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and in various subgroups, were analyzed. Also considered was the 
extent to which observed relationships were influenced by 
nonval idi ty-related factors. Findings provided direct evidence that 
the shortened SLEP and locally developed writing tests were providing 
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ABSTRACT 



This is the report of a study that was underaken to obtain 
direct empirical evidence regarding aspects of the validity and 
usefulness for ESL placement of (a) a shortened version of the 
Secondary Level English Proficiency (SLEP) Test, being used for ESL 
placement by colleges in the Los Angeles Community College District 
(LACCD) , and (b) locally developed and scored writing tests. The 
LACCD Central Office provided scores on the shortened version of 
the SLEP and the writing samples, grades in ESL courses, and 
background data (gender, language, educational status, and so on) 
for over 10,000 students. This report documents and evaluates 

• patterns of performance on the shortened SLEP and the 
writing tests in the general ESL population and in 
selected demographic subpopulations, 

• concurrent relationships among scores on the components 
of the LACCD placement battery, 

• observed levels of correlation between scores on the 
shortened SLEP test, the writing test, and the placement 
composite, on the one hand, and student performance in 
ESL courses, as indexed by grade earned (a grade on the 
"A-F" scale, or a Pass/Fail grade) , on the other, by 
course and by college, and in various subgroups (e.g., 
gender, educational level, age, language) , and 

• the extent to which observed relationships in placed 
samples are influenced by non-validity-related factors 
(for example, differential restriction of range on the 
tests that were used to place students, sample size, and 
type of grading system) . 

The findings of this collaborative undertaking provide direct 
empirical evidence that the shortened SLEP and locally developed 
writing tests are providing valid information regarding related 
aspects of ESL proficiency in the demographically diverse ESL 
student population being served by the LACCD; and the findings 
logically extend available evidence supportive of the validity of 
the Secondary Level English Proficiency Test for ESL assessment 
purposes in the LACCD and elsewhere. Based on study findings, the 
local writing tests and the shortened SLEP appear to be providing 
an effective basis for placing students, within time constraints 
that appear to be considered necessary, from an administrative 
perspective. Research is needed to address questions regarding the 
extent to which use of the full-length version of the SLEP would 
enhance the overall validity of placement. Some pertinent lines of 
inquiry are suggested. 
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EXECUTIVE SUMMARY 



Study Context 

To facilitate placement of students for instruction in English 
as a second language (ESL) in courses differentiated according to 
seven ESL prof iciency levels, colleges in the Los Angeles Community 
College District (LACCD) consider scores on a shortened version of 
the Secondary Level English Proficiency (SLEP) test (e.g., ETS, 
1988, 1991), and locally developed writing tests — writing samples^ 
involving locally selected topics and rated locally, but elicited 
under standard time limits. 

The SLEP was selected by the LACCD ESL Committee, as the 
standardi zed-test component of the placement battery, after a 
rigorous, comparative review of 

(a) the content, psychometric properties and costs of nine 
commerically available ESL proficiency tests (see Appendix A. 2 for 
data on the tests that were considered) , and 

(b) empirical findings (Butler, 1989) indicating that SLEP 
items were at a psychometrically appropriate level of difficulty 
for LACCD ESL students, and that SLEP scores were positively 
related to independently established ESL placement levels. 1 The 
review process and the considerations involved in arriving at 
decisions as to how and to what extent the SLEP would be modified 
for use in the LACCD, are described in detail by Butler (1989) . 



Objectives of the Present Study 

The work reported herein was undertaken with the encouragement 
and support of the SLEP Testing Program at Educational Testing 
Service, and the LACCD central office, represented throughout by 
Ms. Rebecca Tillberg, to obtain direct empirical evidence regard- 
ing aspects of the validity ar.d usefulness for ESL placement of (a) 
the shortened version of the SLEP used in the LACCD, and (b) the 
locally developed and scored writing tests. More specifically, the 
principal objectives of the study were 

(a) to document and evaluate patterns of performance on the 
shortened SLEP and the writing tests in the general ESL population 
and in selected demographic subpopulatior,*;, 



Butler noted that both the SLEP and the Pre-TOEFL, an 
easier version of the familiar Test of English as a Foreign 
Language, were attractive because of the communicative orientation 
of the item types, "consistent with the communicative focus of the 
ESL curriculum. 11 Cost considerations favored the SLEP (Butler 
1989: pp. 4-5). 

LACCD/SLEP Study iii 



erIc 6 



(b) to investigate levels and patterns of concurrent rela- 
tionships among scores on the components of the LACCD placement 
battery, 

(c) to obtain base-line empirical evidence regarding observed 
levels of correlation between scores on the shortened SLEP test, 
the writing test, and the placement composite, on the one hand, and 
student performance in ESL courses, as indexed by grade earned (a 
grade on the "A-F" scale, or a Pass/Fail grade) , on the other, by 
course and by college, and in various subgroups (e.g. , gender, 
educational level, age, language) , and 

(d) to provide an analytic assessment of the extent to which 
observed relationships in placed samples are influenced by non- 
validity-related factors (for example, differential restriction of 
range on the tests that were used to place students, sample size, 
and type of grading system) . 

Given the data at hand, only the validity properties of the 
shortened version of the SLEP and the writing test are illuminated 
directly by this investigation. 

• However, the full-length version of the SLEP can be 
expected to exhibit in the LACCD context, levels of 
concurrent, discriminant, and criterion-related validity 
equal to or greater than the levels that may be found to 
obtain for the shortened version of the SLEP that is 
currently being used in the LACCD. 

Thus, an important incidental objective of the study was to 
extend available evidence of the validity of the full-length SLEP 
test. 



General Analytical Rationale 

A general outline of elements in the overall analytical 
rationale that guided inquiry involving the respective objectives 
is provided below. 

Concurrent and Discriminant Validity 

The placement tests (shortened LC, shortened RC, Essay) are 
designed to measure listening, reading, and writing skills, 
respectively, thought of as distinguishable but related aspects of 
general ESL proficiency. 

• Moderate to moderately high levels of intercorrelation 
among such measures would constitute evidence of con- 
current validity. 
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• If the shortened SLEP reading score is found to be more 
closely correlated with Essay rating than is the short- 
ened SLEP listening score, this would constitute evidence 
of discriminant validity for the shortened version of the 
SLEP, namely, evidence that the LC and RC sections are 
measuring psychometrically distinguishable aspects of 
proficiency. 

Implications of Concurrent Validity Findings 
for Reliability 

It was not feasible to address directly guestions regarding 
the reliability of either the shortened SLEP or the local writing 
tests in the specific samples under consideration. 

• Internal consistency reliability estimates for item- 
type subsections of the full-length SLEP are available 
for samples involved in test development--. 89 and .81, 
respectively, for the two item- type subsections included 
in the shortened LC section, and .71 and .89, respec- 
tively, for the two reading item-type sections in the 
LACCD (shortened) version. 

• Based on available estimates and by inference from de- 
crease in length, it is assumed that the shortened SLEP 
sections are somewhat less reliable than their full- 
length counterparts: internal consistency estimates of 
.94 for LC and .93 for RC (.96 for total). 

• Inter-rater reliabilities for the writing test are 
assumed to be lower than those for the shortened SLEP, 
and perhaps not consistent across rating sites, but no 
data on inter-rater reliability are available for 
citation. 

However, it is important to keep in mind that useful general 
reliability-related inf erences can be drawn from observed correl- 
ations among variables. 

• Because the level of correlation between tests is lim- 
ited by their respective reliabilities, if either the 
shortened SLEP or the writing test has very low reliabil- 
ity, generally speaking, the scores involved cannot cor- 
relate very highly with each other or with any other 
study variable. Hence, moderate to moderately strong 
concurrent validity coefficients, for example, are in- 
directly indicative of "useful" levels of reliability. 

Problems Specific to the Writing Tests 

The possibility of systematic differences across colleges with 
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respect to "rater bias" is inherent in any context involving the 
generation of ratings in multiple sites, even though a common scale 
is employed. Butler (1989) called attention to anomalous differ- 
ences, by college, in the average essay ratings of students at the 
same ESL placement level. 

• Such differences may reflect substantive differences in 
the average level "writing ability" of the students being 
assessed on the respective campuses, but systematic dif- 
ferences in rating standards may also be involved. SLEP 
(objective test) scores, of course, are free of this type 
of bias. 

• The presfciice of statistically significant differences 
by college in level of Essay rating relative to SLEP 
performance, would constitute a necessary condition for 
inferring differences in rating standards. 

Differences in all correlational findings involving 
writing samples may to some extent be associated with 
differences across colleges in rater-reliability, topics 
selected, and so on. 

Need to Investigate Properties of Placement Composites 

Results of the foregoing lines of inquiry bear directly on the 
validity properties of the shortened SLEP and the local writing 
tests. In practice, the information provided by the three measures 
(two SLEP scores and the Essay rating) is combined using nominal 
weights for percent-right transformations of the raw scores on the 
shortened LC and RC sections, respectively, and the Essay. The 
actual contribution of two or more variables to a composite does 
not necessarily correspond to the nominal weights because of dif- 
ferences in score variabilities. 

• In samples from City, Harbor, Mission, Trade, and 
Valley, some two-thirds of the variance in the placement 
composite should be associated with SLEP performance, and 
one-third with essay performance (nominal weights for 
Essay, LC and RC were, respectively, .34, .33, and .33, 
in samples under consideration) . 

• In samples from East and Pierce, some 60 percent of 
composite variance should be associated with essay per- 
formance, and only 40 percent with SLEP performance (nom- 
inal weights for Essay, LC and RC were .60, .20, and .20, 
respectively) . 

Regression-based estimates of the relative contribution of the 
test scores to composites were obtained for each college, and 
compared with the expected relative contribution, based on the 
nominal weights. 
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Considerations in Assessing Criterion-Related Validity 

In the LACCD context, all tests used are expected to exhibit 
"reasonable" levels of correlation with appropriate criteria of 
performance in the courses of study to which the testing is 
related — average criterion-related validity coefficients in the 
neighborhood of .35 have been suggested as representing a reason- 
able standard (California Community Colleges, 1992: p. 20) . 

Although a "standard" is proposed, it is specifically 
recognized that 

• "the magnitude of the correlation may vary as a func- 
tion of the degree to which a tests was used to place 
students in the course under investigation and/or the 
variation in grading standards across classrooms" (Cali- 
fornia Community Colleges, 1992: p. 20). 

Empirical evidence based on LACCD samples, regarding typical 
levels and ranges of test-criterion correlation — either for current 
placement tests or similar tests — does not appear to be available. 
Accordingly, analysis of test/grade correlations in the present 
study was designed primarily to 

• generate base-line, empirical findings regarding levels 
and patterns of test/grade correlation for the shortened 
SLEP and the writing test in the LACCD context, and 

• assess the extent to which effects associated with use 
of the tests in placement, influence both levels and 
patterns of observed test/grade correlations in ways that 
limit generalization from the observed correlations 
regarding test validity in the general ESL population 
being assessed. 



Study Sample, Data, and General Procedure 

The foregoing objectives were pursued, usinj data provided by 
the LACCD Central Office, for more than 10,000 students who were 
assessed for placement in Spring 1991, Fall 1991 and Spring 1992, 
respectively. Data were available for City, East, Harbor, Mission, 
Pierce, Trade, Valley and West — all LACCD colleges except South- 
west. 

Student records contained scores on the placement tests, 
scores on composites computed for placement on the respective 
campuses, data needed to classify students according to specific 
ESL courses and corresponding subject areas, official end-of -course 
grades (e.g., A-F, Pass/Fail, Withdrew), and self-reported back- 
ground data: gender, educational status (e.g., less than high 
school graduate, high school graduate, college graduate) , ethnicity 
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(national origin and ethnic group) , date of birth, and native 
language (11 categories, of which nine were specific languages) . 

• Information provided in college catalogs (especially 
from City) helped guide classification of courses by ESL 
level . 

• Analyses concerned with objectives other than those 
requiring the assessment of test/grade relationships, 
were conducted using data for the total (unrestricted, 
pre-placement) samples from the respective colleges. 

• Only students with clearly quantifiable grades (that 
is, a grade of A, B, C, D, or F, or a Pass/Fail grade), 
in courses represented by at least 10 students with a 
grade and scores on the placement tests, were included in 
analyses concerned with criterion-related validity issues 
(test/grade correlations) . 



Organization of the Main Report 

Section I of the study report provides a general overview of 
the process through which the foregoing placement tests were ident- 
ified and selected, (b) generally reviews evidence bearing on the 
validity of the full-length SLEP in various use contexts, including 
the LACCD) , (c) reviews the considerations involved in decisions to 
use only a portion of the full-length SLEP test and in selecting 
particular SLEP items for inclusion in the shortened version, and 
(d) provides a brief overview of study objectives and the data 
provided by the LACCD central office. 

Section II provides a general description of the performance 
of the general ESL student population and several demographic sub- 
populations on the shortened SLEP and the writing tests. 

Section III reports findings regarding concurrent- and 
discriminant-validity properties of the shortened SLEP and the 
locally developed writing tests, along with findings bearing on the 
comparability of the writing tests across colleges. Results of 
analyses of the relative contribution of SLEP scores and writing 
test scores in placement composites are also reported in Section 
III, along with illustrative evidence of the extent to which use of 
tests in placement introduces range-restriction-related effects 
that complicate validity-related interpretations of correlation 
coefficients computed within samples of placed students. 

Section IV reports and evaluates findings regarding test/grade 
correlations by course, by college, and for subgroups defined by 
gender, language, educational status, and age. The extent to which 
the observed coefficients are influenced by nonvalidity-related 
variables (sample standard deviations, sampla size, and type of 
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grading system) is demonstrated. 



Section V provides a general review and evaluation of the 
findings. 



Overview of Study Findings 

The ESL population being assessed with the shortened SLEP is 
extremely heterogeneous with respect to age, educational level, 
ethnicity, and language background. 

• The average student is about 29 years of age; based on 
data of record on date of birth, some students were less 
than 13 years old and others were over 65 years of age 
when assessed in the LACCD. 

• The majority of students hold secondary-school diplomas 
earned in schools located outside the United States, but 
the sample includes individuals who have not completed 
secondary school — some having enrolled in adult education 
courses, or classes for special students — as well as 
individuals with bachelor's or higher degrees. 

• The largest linguistic subpopulation is made up of 
native speakers of Spanish (about 44 percent of the 
total) ; none of the other directly identified language 
groups accounts for much more than 10 percent (11 percent 
reported Armenian, and less than 1 percent reported 
Filipino) . 

The shortened SLEP sections are at psychcmetrically efficient 
levels of difficulty for the general LACCD ESL population as well 
as for subgroups that differ markedly with respect to age, edu- 
cational level, language background, and self-reported ethnic group 
membership . 

• Results of an analysis of trends in mean scores on the 
shortened SLEP and the Essay, respectively, across 
subgroups classified by educational level indicate that 
the two means covaried directly. Similar patterns of 
covariation are discernible for the two test means across 
other demographic subgroups — that is, demographic 
subgroups with higher (lower) means on the SLEP tend to 
have higher (lower) means on the writing test. 

Concurrent relationships between shortened SLEP scores and 
scores on the writing tests were moderate to strong in each college 
setting : 

• within-college correlations between scores on the 
locally developed writing tests and scores on the 
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shortened SLEP centered at about .60, and varied between 
roughly .5 and .7 across colleges. These correlations 
were observed in general (pre-placement) samples. 

• The relationship between the shortened SLEP reading 
comprehension score and Essay rating typically was 
somewhat stronger than that observed for the shortened 
SLEP listening comprehension score with Essay — consistent 
with plausible expectation that reading and writing 
skills, should tend to be somewhat more closely related 
than are listening and writing skills. 

• At the same time, coefficients for total score on the 
shortened SLEP test (LC + RC) were somewhat larger than 
those for RC alone, indicating that each of the two sec- 
tions may be providing some unique information regarding 
aspects of the ability being measured by the writing 
test — a face valid measure of ability to write compre- 
hensibly in English. 

• Levels of conconcurrent relationship observed for 
college-level samples were also observed in the results 
of analyses conducted by college, illustratively, for 
Spanish-speaking and Korean-speaking subgroups, 
respectively . 

The foregoing findings provide strong empirical evidence of 
concurrent- and discriminant-validity properties for the shortened 
SLEP and the writing tests. 

In this, connection, it is considered particularly noteworthy 
that the levels of concurrent relationship observed in the LACCD 
context, between scores on the shortened version of the SLEP and 
scores on local writing tests (with attendant differences in topic, 
rating procedures, and so on) , are comparable to levels that have 
been found to obtain between scores on the Test of English as a 
Foreign Language and the Test of Written English (writing samples 
elicited under standard conditions, scored under controlled 
conditions by at least two raters) , 

(a) in samples tested in developmental research (Carlson, 
Bridgeman, Camp, and Waanders, 1985) involving the TOEFL and 
prototypical versions of the TWE, and 

(b) in samples taking both tests under fully operational 
conditions (see ETS, 1992b). Aaong other things, such findings 
Attest to the power of direct observation of pertinent linguistic 
behavior in ESL proficiency assessment. 

Analyses of levels and patterns of correlation of the 
placement tests with course grades, involving 59 course-level 
samples, were complicated due to the need to assess the influence 
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of nonvalidity-related factors (for example, restriction of range 
effects) on both levels and patterns of observed test/grade coef- 
ficients. The generalizations outlined below reflect findings 
reviewed in detail and evaluated in Section 4. 

• Scores on the shortened SLEP and the writing test, re- 
spectively, and as combined for placement purposes in 
different colleges, tend to be positively correlated with 
course grade, but there is substantial variability across 
courses in level of relationship. Distributions of ob- 
served course-level coefficients for the respective tests 
tend to center at about the .2 level, with substantial 
variability (ranging upward from negative values to 
values above . 5 ) . 

• Regression results indicated substantial agreement 
between actual and nominal weights for the placement 
tests in composites. 

• Essay scores received relatively more weight than did 
SLEP scores in placement at East and Pierce, and in sam- 
ples from those colleges, average coefficients were high- 
er for the SLEP than for the Essay. SLEP was weighted 
relatively more heavily than Essay in other college sam- 
ples, and in these samples, Essay coefficients typically 
were higher than SLEP coefficients. Such findings are 
interpretable as reflecting differential restriction of 
range effects. 

• Differences across 59 course-level samples, with re- 
spect to level of test/grade coefficients for the place- 
ment tests (that is, SLEP/grade coefficients, Essay/grade 
coefficients, and Composite/grade coefficients, respec- 
tively) , were found be associated relatively strongly 
with nonvalidity-related sample characteristics: dif- 
ferences in the corresponding sample standard deviations, 
type of grading system employed (A/F versus Pass/Fail) , 
and sample size. 

The respective test/grade coefficients for the 59 course-level 
samples, treated as dependent variables, were regressed on the five 
nonvalidity-related variables noted above, in order to assess their 
relative contribution when treated as a battery of predictors. 

• Sample standard deviation for the shortened SLEP was 
the most highly weighted variable not only in analyses 
involving course-level SLEP/grade coefficients but also 
in analyses involving Composite/grade coefficients. 

• The most, important predictor of level of Essay/grade 
correlation was type of grading system; SLEP standard 
deviation was the second largest predictor. 
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• The Composite standard deviation did not contribute to 
level of Essay/grade correlation in the stepwise regres- 
sions, after other variables were entered. 

Regression equations developed in the foregoing analysis 
provided a basis for estimating test/grade coefficients (SLEP/ 
grade, Essay/grade, and Composite/ grade coefficients, respec- 
tively) , assuming no selection on the variables involved . Values 
for the total LAC CD (general) sample standard deviations were 
substituted for the restricted values in the respective equations. 

Estimated coefficients and the corresponding observed 
coefficients are shown below. 

Test/grade correlations 

Short SLEP Essay Composite 



A/F 


sys 


(est. ) 


.47 


.35 


.50 


A/F 


sys 


(obs. ) 


.17 


.20 


.22 


P/F 


sys 


(est.) 


.43 


.21 


.42 


P/F 


sys 


( obs . ) 


.18 


.07 


.19 



Generally speaking, the findings involving test/grade rela- 
tionships that have been reviewed should be thought of primarily as 
a basis for empirical assessment of the extent to which observed 
course-level test/grade coefficients generally are irfluenced by 
nonvalidity-related factors, and are thereby limited as bases for 
inferences regarding "levels or patterns of validity" for the tests 
involved when applied in the general population under consider- 
ation. 

In analyses for subgroups, positive test/grade correlations 
varying around the observed values noted above were found to obtain 
in samples aggregated by gender, age, educational level, and 
language, respectively. 

Regarding the Writing Tests 

• Pooled within-school SLEP/Essay coefficients typically 
were larger than those observed for the combined sample, 
the latter being attenuated by inconsistent ordering of 
school means on the respective tests — an anomalous find- 
ing, in view of the systematic, positive within-school 
relationships observed. 

• College differences in "rating standards" are suggested 
by the results of analyses of differences across colleges 
with respect to mean discrepancies between observed Essay 
rating, and Essay rating estimated from SLEP performance, 
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using a total-sample regression equation. Mean discrep- 
ancies (residuals') were substantially more pronounced in 
analyses by college than in analyses for subgroups by 
gender, language, educational level, and age, respec- 
tively, aggregated across colleges. 

• In view of apparent differences across sites in "rating 
standards," pending resolution of questions regarding 
comparability of Essay scores across colleges, future 
investigations of subgroup differences in the relative 
development of Essay-assessed "writing ability" and SLEP- 
assessed skills, need to be conducted at the college 
level . 

These and related issues constitute meaningful topics for 
further inquiry. The findings suggest the need for systematic 
study of the rating process on each campus, including the collec- 
tion of data needed to evaluate degree of agreement among raters 
with respect to level as well as rank-order. 



Concluding Observations 

The findings of this collaborative undertaking provide direct 
empirical evidence that the shortened SLEP and locally developed 
writing tests, are providing valid information regarding related 
aspects of ESL proficiency in the demographically diverse ESL 
student population being served by the LACCD; and the findings 
logically extend available evidence supportive of the validity of 
the Secondary Level English Proficiency Test for ESL assessment 
purposes in the LACCD and elsewhere. 

Considered in light of the rigorous screening and evaluation 
of available psychometric options that resulted in SLEP's initial 
selection by the ESL Committee, and other evidence of SLEP's val- 
idity cited herein — including the generally positive findings re- 
ported by Butler (1989) , regarding SLEP's appropriateness for ESL 
assessment purposes in the LACCD — the findings that have been 
reviewed herein both support and extend the working proposition 
advanced at the outset, namely, that 

• the SLEP is a valid test of psychometrically distinct, 
albeit closely related, aspects of acquired proficiency 
in English as a second language: the ability to com- 
prehend utterances in English, and the ability to read 
and comprehend the substance of material written in 
English, 

• the SLEP can be expected to provide reliable and valid 
information regarding these abilities, when used for ESL 
assessment purposes by secondary schools, colleges, and 
other institutions, and 
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• given the demographic diversity of the LACCD samples 
involved in this study, the findings extent available ev- 
idence indicating that the SLEP can be used validly with 
local ESL student populations that are quite heterogen- 
eous with respect to age, educational level, language 
background, and national origin. 

The local writing tests and the shortened SLEP appear to be 
providing an effective basis for placing students, within time 
constraints that appear to be considered necessary, from an 
admini strat ive perspective . 

As noted by Butler (1989), some cost in terms of diminished 
reliability and validity undoubtedly is entailed by the use of only 
a selected portion of the test. Available evidence bearing on this 
important question is limited, but consistent with the validity- 
cost assumption. 

• In a cooperative study currently in progress (Wilson, 
1993), involving large samples of Japanese- speaking stu- 
dents being assessed for ESL placement at Temple Univers- 
ity-Japan, LACCD-parallel subscores were computed post 
hoc from data available for the full-length SLEP. Cor- 
relations between these scores and ratings of speaking 
ability and writing ability, respectively, were approx- 
imately .05 correlation points lower than correlations 
for corresponding full-length scores, that centered 
around . 6 . 

• Rudmann (1991), at Irvine Valley College, reports ob- 
s erved correlations centering around .40 between grades 
in ESL courses and scores on the full-length SLEP. 

Analyses involving the full-length SLEP test clearly would be 
needed in order to address questions of comparative validity: com- 
parative validity of the current shortened, LACCD version of the 
SLEP, and the full-length test, and/or questions as to the com- 
parative validity of the particular SLEP item types selected versus 
those not selected. 2 



2 In connection with the latter issue, it is noteworthy that 
one LACCD college (West) opted to use only the shortened SLEP 
reading section, with the writing test, for placement. More 
generally, in reporting on the basic SLEP validation study, 
Stansfield (1984) observed that users might elect to test either 
listening comprehension or reading comprehension in some 
circumstances (e.g., listening comprehension for level-placement 
decisions involving courses emphasizing the development of 
conversational skills) . An evaluation of experience at West, where 
only a score on items from the SLEP "reading comprehension" ability 
domain is used, would be useful. 
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SECTION I. CONTEXT OF THE STUDY 

The nine colleges that make up the Los Angeles Community 
College District (LACCD) offer a wide range of courses in English 
as a second language (ESL) for nonnative-English speaking students 
and prospective students. The colleges are designated herein as 
City, East, Harbor, Mission, Pierce, Southwest, Trade, Valley, and 
West, respectively. The programs offered by these colleges conform 
generally to a curricular model that was developed by the LACCD ESL 
Committee during 1987-88, calling for placement of students in one 
of seven proficiency levels, based on results of placement testing 
and other pertinent considerations (Butler, 1989) . As described by 
Butler, the model did not incorporate the use of a standard exit 
test — for example, alternate forms of the placement test. 

Since 1988, placement of students according to ESL levels has 
been guided by scores on a shortened version of the Secondary Level 
English Proficiency (SLEP) test (e.g., ETS , 1988, 1991), and 
locally developed writing tests — writing samples, involving locally 
selected topics and rated locally, but elicited under standard time 
limits. 

The work reported herein was undertaken, using data provided 
by the LACCD central office, to obtain empirical evidence regarding 
aspects of the validity and usefulness for ESL placement of (a) the 
shortened version of the SLEP used in the LACCD — and logically to 
extend the body of available empirical evidence bearing on the 
validity of the full-length SLEP test — and (b) the locally 
developed and scored writing tests. 1 

This section (a) provides a description of the process through 
which the foregoing placement tests were identified and selected, 
(b) generally reviews evidence bearing on the validity of the full- 
length SLEP in various use contexts, including the LACCD), (c) 
reviews the onsiderations involved in decisions to use only a 
portion of the full-length SLEP test and in selecting particular 
SLEP items for inclusion in the shortened version, and (d) provides 
a brief overview of study objectives and the data provided by the 
LACCD central office. 

Section II provides a general description of the performance 
of the general ESL student population and several demog: »phic 
subpopulations on the shortened SLEP and the writing tests. 
Section III reports findings regarding concurrent- and dis- 
criminant-validity properties of the shortened SLEP and the locally 
developed writing tests, along with findings bearing on the 



The work was undertaken with the encouragement anr> support 
of the SLEP Testing Program at Educational Testing Service, and 
the LACCD central office, represented throughout by Ms. Rebecca 
Tillberg. 
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comparability of the writing tests across colleges. Results of 
analyses of the relative contribution of SLEP scores and writing 
test scores in placement composites are also reported in Section 
III, along with illustrative evidence of the extent to which use of 
tests in placement introduces range-restriction-related effects 
that complicate validity-related interpretations of correlation 
coefficients computed within samples of placed students. 

Section IV reports and evaluates findings regarding test/ grade 
correlations by course, by college, and for subgroups defined by 
gender, language, educational status, and age. The extent to which 
the observed coefficients are influenced by non-validity related 
variables (sample standard deviations, sample size, and so on) is 
demonstrated. Section V provides a general review and evaluation 
of the findings. 



Placement Testing in the LACCD 

Placement testing in the LACCD involves the collection of a 
writing sample and administration of a shortened version of the 
Secondary Level English Proficiency (SLEP) test — a standardized, 
multiple-choice test with sections measuring the ability to compre- 
hend utterances in English (listening comprehension) and the abil- 
ity to comprehend written material in English (reading comprehen- 
sion) , respectively, in samples of nonnative-English speakers (for 
example, Educational Testing Service [ETS] , 1988, 1991). Brief 
descriptions of the writing test and the full-length SLEP, the 
process through which the SLEP was originally selected, and the 
procedures involved in shortening the SLEP, are provided below. 

The Writing Sample. Generally speaking, writing samples have 
face validity as measures of ability to write comprehensibly in 
English. Problems involved in the use of writing samples have to 
do with the subjective nature of the process through which the 
samples are scored, difficulty in obtaining a representative sample 
of writing skills, and so on. 2 For example, the rank-ordering of 
samples accoz ling to a particular rating scale may differ from 
rater to rater; and raters giving similar rank-orders to a set of 
samples may differ systematically with respect to scale-level 
assignments; performance may not be consistent across topics. 

In the LACCD context, writing samples are collected and scored 
locally; essay topics are also locally developed. However, samples 
are scored according to a common, holistic rating scale, involving 
eight defined levels ranging from 0 thru 7 (see Appendix A.l). 



For a comprehensive review of problems involved in assessing 
"writing ability" in samples of ESL speakers, and the complex pro- 
cedures required to deal with these problems, see Carlson, 
Bridgeman, Camp, & Waanders (1985) . 
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According to Butler (1989: p. 7) , " ... the scale was developed 
by Dr. Minette Lanier, Pierce College, and Dr. Lloyd Thomas of the 
LACCD District Office, with suggestions made by members of the ESL 
Committee. 11 

Studies designed to assess the reliability and validity of 
writing samples do not appear to have been conducted in the LACCD 
context. However, Butler reported 11 . . .a wide range of (average) 
performance in writing for the same (ESL placement) level at dif- 
ferent colleges" (1989: p. 18), citing relatively marked differenc- 
es between average essay ratings for Pierce (typically about "5" on 
the holistic scale) and Mission (typically about "3" on the 
scale). Differences in average ratings across "scoring sites," 
may reflect substantive differences in writing skills, but site 
differences in "rating standards" cannot be ruled out as contrib- 
uting factors. Raters at some sites may tend to rate given samples 
higher (lower) than do raters at other sites. 

The SLEP . The SLEP was developed by the Educational Testing 
Service for use by secondary-level institutions in screening and/or 
placing nonnative-English speaking applicants, to be tested in ETS- 
operated centers worldwide. When the latter practice was discon- 
tinued, the SLEP was made available to qualified users for local 
administration and scoring. 

The basic validation study (Stansfield, 1984) that accompanied 
introduction of the SLEP, provided evidence that SLEP scores were 
related positively to ESL placement classification, years of study 
of English as a second language, time spent in the U.S., and other 
criteria, in samples from over 50 U.S. secondary schools. Moderate 
to relatively high concurrent relationships have been reported 
(e.g., ETS, 1991) for SLEP scores with scores on the Test of En- 
glish as a Foreign Language [TOEFL] (e.g., ETS, 1992a), used pri- 
marily with college-level ESL students, and the Maculaities As- 
sessment Program (Maculaities, 1982) , used primarily with pre- 
college-level students (see Von Schilling, 1988) . 

The SLEP is currently being used worldwide, in a variety of 
assessment contexts, with both traditional and nontraditional stu- 
dents, not only in secondary-school settings but also in post- 
secondary settings. Approximately one-third of the respondents to 
a survey of SLEP users (Wilson, 1993) reported use of the SLEP with 
college-level students, or students in adult education classes: to 
assess readiness to undertake English-medium academic instruction, 
for placement in ESL courses, for course or program evaluation, 
admission screening, and so on. 

Face validity considerations, empirical evidence of SLEP's 
psychometric properties, such as that cited illustratively above, 
and positive reports of SLEP's validity-related properties from ESL 
professionals using the SLEP in secondary and postsecondary 
settings, constitute what appears to be a conceptually persuasive 
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basis for asserting as a working proposition, that 

• the SLEP is a valid test of psychometrically di: tinct, 
albeit closely related, aspects of acquired proficiency 
in English as a second language, namely, the ability to 
comprehend utterances in English, and the ability to read 
and comprehend the substance of material written in 
English, and can be expected to provide reliable and 
valid information regarding these abilities in samples of 
literate ESL speakers at relatively diverse educational 
levels. 

The SLEP was selected as the standardi zed-test component of 
the placement battery, based on a comprehensive, comparative review 
by the ESL Committee of the content, psychometric properties, and 
costs of nine commercially available ESL proficiency tests (see 
Appendix A. 2 for data on the tests that were considered) . 3 The 
review process and the considerations involved in arriving at 
decisions as to how and to what extent the SLEP would be modified 
for use in the LACCD, are described in detail elsewhere (Butler, 
1989) . As noted specifically from time to time, descriptions of 
the process herein are based on Butler's detailed report. 

Shorten ing the SLEP . The SLEP includes a total of 150 mul- 
tiple-choice questions in two sections — listening comprehension and 
reading comprehension — of 75 items each. The time required for the 
entire test is approximately 85 minutes: just under 40 minutes, 
paced by recorded prompts, for the listening section and 45 minutes 
for the reading section. Reusable test booklets are available for 
three equated forms of the SLEP (Forms 1, 2, and 3). To date, only 
Form 1 has been used in the LACCD (Tillberg, personal communica- 
tion, 1992). 

Eight different item types are represented in the SLEP, four 
in the listening section and four in the reading section, as fol- 
lows: Listening section (Single Pictures, Dictation, Map, Conver- 
sations), and Reading section (Cartoons, Four Pictures, Cloze, and 
Reading Passage) . Illustrative items are provided in Appendix A. 3. 

The decision to use a shortened version of the SLEP test was 
dictated primarily by time considerations — the total amount of time 
required for both the SLEP and a writing sample was deemed to be 



Butler noted that both the SLEP and the Pre-TOEFL, an 
easier version of the familiar Test of English as a Foreign 
Language, were attractive because of the communicative orientation 
of the item types, "consistent with the communicative focus of the 
ESL curriculum." Cost considerations favored the SLEP (Butler, 
198y: pp. 4-5) . 
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excessive. 4 To obtain empirical evidence regarding the performance 
of LACCD ESL-students on the SLEP, and to evaluate performance on 
subscores based on SLEP item types, Butler (1989) analyzed item- 
type and section-level scores for the entire test, in Spring 1988, 
for samples from the nine LACCD campuses. 

Information regarding characteristics of the LACCD samples in- 
volved (e.g., age, educational status, gender, language group) was 
not reported. However, it is pertinent to note that the distribu- 
tions of percent-right scores for the respective test-sections were 
judged by Butler (1989: p. 25) to indicate that the SLEP items were 
not inappropriately "easy" or "difficult" for LACCD samples in 
various proficiency levels. 

Data shown in Table 1.1 indicate that average percent-right 
scores for LACCD samples studied by Butler were generally com- 
parable to those reported by ETS (internal memoranda) for the 
samples of international students involved in the development of 
different forms of the SLEP. Table 1.1 includes average percent- 
right scores for a sample of native-English speakers (G7-12 samples 
from several Florida schools, tested by Holloway [1982]). Also 
shown in the table are estimated internal consistency reliability 
coefficients for the respective SLEP item-type subsections, from 
unpublished internal ETS test analyses (see Attachment A) . 

o SLEP items are very easy for native-English speakers, and 
judging from the LACCD data, for example, they become 
increasingly more difficult as the independently assessed 
level of English proficiency of the test-taking sample 
decreases — suggesting SLEP's validity as a measure of 
proficiency in English as a second language generally, as 
well as in samples from the LACCD context. 

Selecting item types for the shortened test. As reported by 
Butler (1989) , decisions regarding item types to be included in the 
shortened version were guided by feedback from the respective 
campuses regarding the perceived appropriateness of item content, 
and the extent to which scores based on the respective item type 5 
discriminated among the several proficiency levels represented in 
the sample; also the progression in difficulty for the four reading 
item types in the LACCD sample — see data for last fou:> item types 
in Table 1.1, for all the samples involved. After evaluating all 
considerations, the ESL Comittee decide to use a version of the 
SLEP that included four of the eight item- type sections, namely, 
Single Picture and Dictation item types from the Listening section, 



As reported by Butler (1989: p. 10), "... word came back 
from every campus that the counselors would be extremely reluctant 
to agree to what they considered to be a lengthy (2 hour) testing 
period" (p. 10) . 
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Table 1.1. Comparative Performance of LACCD Samples and Designated 
El and E2 Samples on SLEP Item-Type Subsections: Mean 
Percent Right Scores 



SECTION/ Samples* 

Estimated 
El E?a E2b E2c reliability 
Min Max 

LISTENING 



Single picture 97 85 

Dictation 93 90 

Map 97 74 

Extended con- 93 68 
versations 



81 


(53 


83) 


.811 


84 


(48 


87) 


.889 


69 


(44 


76) 


.721 


62 


(35 


69) 


.840 



READING 



Cartoon 

Four pictures 

Cloze 

Literary pas- 
sage 



94 


89 


87 


91 


78 


84 


81 


62 


62 


64 


48 


56 



(63 


91) 


.742 


(55 


82) 


.707 


(29 


68) 


.891 


( 6 


43) 


.685 



Mote. E1 is used to denote English as the first or prinary language; E2 is used to denote English 
as a second language. 

* E1 data are from Holloway (1934), total sample (G7-12) of students from selected Florida schools. 
E2a and E2b data are from ETS unpublished internal analyses for international -student samples in 1980 
<EZa) and 1988 (E2b) used in test development and equating. The E2a data are for Form 1 of the SLEP, 
the form being used in the LACCD. E2c data are from Butler (1989), for samples of ESL students from 
LA„CD colleges. The minimum (Hin) percent-right scores are for ESL students in the most elementary of 
seven ESL levels defined for the system; the maximum (Max) percent-right means are for those at the 
highest level. 



** From unpublished internal test analyses (ETS, 1980). 



and Four Picture and Cloze items from the Reading section. Time 
limits and item composition of the shortened SLEP and the writing 
sample are indicated in Table 1.2. Butler (1989) called attention 
to the potential loss of placement-related information associated 
with the elimination of a substantial number of SLEP items, and 
noted that "... it may be necessary to reevaluate the choice of 
SLEP Test subsections" (p. 2). 
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Table 1.2 Components of the Recommended Placement Battery- 



Test 



No. items Time 

(mins. ) 



Writing Sample 



1 



30 



Short SLEP 
Listening 



45 
25 
20 



20 



One Picture 
Dictation 



Reading 



35 



Four Pictures 
Cloze 



15 
40 
100 



Total (short SLEP) 



85 



Note . See Appendix A. 3 for illustrative Items. 



Guidelines were developed for using the tests in placement, 
including (a) use of a "standard" placement composite, and (b) 
recommeded composite-score ranges for proficiency levels, to 
facilitate uniform placement practices across colleges. 

The recommended standard placement composite was to be derived 
using "nominal" weights of .34 (Essay %rite) , .33 (LC %rite) , and 
.33 (RC %rite) . 5 Score ranges for seven proficiency levels were 
suggested. It was recommended that "additional information such as 
length of time in the United States, number of years of instruction 
in English, educational goals, etc., should also be taken into con- 
sideration in determining where a student can best benefit from 
language instruction" (Butler, 1989, p. 38). 

Colleges were free to modify the weighting procedure that was 
suggested. And, based on information provided by the LACCD office 
(Tillberg, personal communication, 1992) , two colleges (East and 
Pierce) subsequently decided to use nominal weights of .60, .20, 
and .20, respectively, for percent-right scores on Essay, 
Listening, and Reading. A third college (West) initially used the 
standard (.34, .33, .33) weighting for components of the full 



The effective (as opposed to nominal) weighting of two or 
more variables in a composite will vary as a function of the 
dispersions and intercorrelations of the variables involved. "If 
we really want to weight tests in a battery equally we should apply 
to each one a weight inversely proportional to its standard dev- 
iation. Without information as to the validities of the tests and 
of their intercorrelatxons, that would be a reasonable thing to do" 
(Guilford, 1965, p. 424). Such information was not available for 
the LACCD sample studied by Butler (1989) . 
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placement battery, but subsequently decided to use only the Reading 
score and the Essay score in the placement composite, with nominal 
weights of .60 ror percent-right scores on Essay and .40 for 
percent-right scores on Reading. 



The Present Study 

According to information provided by the LACCD District Office 
(Tillberg, 1992, personal communication), informal feedback from 
member colleges suggests that those concerned with ESL placement, 
by and large, have been reasonably satisfied with the placement 
procedures that include scores on the test battery described gen- 
erally above. Of course, the extent to which the positive reaction 
indicated by this informal feedback is "representative," can only 
be determined by a formal assessment of degree of satisfaction with 
the placement procedures — among students as well as among teach- 
ers. Such an assessment is beyond the scope of the present 
inquiry, which is concerned with assessing selected validity- 
related properties of the current components of the placement 
battery — that is, the shortened SLEP and the writing test — in 
samples of ESL students from eight LACCD colleges: City, East, 
Harbor, Mission, Pierce, Trade, Valley, and West. 

More specifically, the study was undertaken with the encour- 
agement and support of the SLEP Testing Program at Educational 
Testing Service, and the collaboration of the LACCD District 
Office, with the following objectives: 

(a) to examine patterns of performance on the shortened SLEP 
and the writing tests in the general ESL population and in selected 
demographic subpopulations ; 

(b) to investigate levels and patterns of concurrent rela- 
tionships among scores on the components of the LACCD placement 
battery ; 

(c) to obtain base-line empirical evidence regarding observed 
levels of correlation between scores on the shortened SLEP test, 
the writing test, and the placement composite, on the one hand, and 
student performance in ESL courses, as indexed by grade earned (a 
grade on the "A-F" scale, or a Pass/Fail grade) , on the other, by 
course and by college, and in various subgroups (e.g., gender, 
educational level, age, language), and 



To the extent that shifts in course-placement based on 
teacher (and/or student) perceptions of "misplacement" early in a 
term are permitted, incidence of change in courses would provide a 
pertinent element in an overall assessment of the adequacy of 
placement . 
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(d) to provide an analytic assessment of the extent to which 
observed relationships in placed samples are influenced by non- 
validity-related factors, especially differential restriction of 
range on the tests that were used to place students. 

Given the data at hand, only the validity properties of the 
shortened version of the SUSP can be illuminated directly by this 
investigation. However, on logical grounds, the full-length 
version of the SLEP can be expected to exhibit, in the LACCD 
context, levels of concurrent, discriminant, and criterion-related 
validity equal to or greater than the levels that may be found to 
obtain for the shortened version of the SLEP that is currently 
being used in the LACCD. Accordingly a more general objective of 
the study is to extend evidence of the validity of the full-length 
version of the test. 

Questions of Concurrent and Discriminant Validity 

The placement tests (LC, RC, Essay) are designed to measure 
listening, reading, and writing skills, respectively, thought of as 
distinguishable but related aspects of general ESL proficiency. 

If moderate levels of intercorrelation among the meas- 
ures are found to obtain, this would constitute evidence 
of their concurrent validity. 

If the shortened SLEP reading score is found to be more 
closely correlated with Essay rating than is the 
shortened SLEP listening score, this would constitute 
evidence of discriminant validity for the shortened 
version of the SLEP, namely, evidence that the LC and RC 
sections are measuring distinguishably different aspects 
of proficiency. 

It was not feasible to address directly questions regarding 
the reliability of either the shortened SLEP or the local writing 
tests. Internal consistency reliability estimates for item-type 
subsections of the SLEP were reported above (see Table 1.1), for 
the SLEP form currently in use in the LACCD. 

Based on available estimates and by inference from 
decrease in length, it is assumed that the shortened SLEP 
is not as reliable as is the full-length version: in- 
ternal consistency estimates are .939 for LC, .930 for 
RC, and .962 for the full length total score, for Form 1. 

Inter-rater reliabilities for the writing test are 
assumed to be lower than those for the shortened SLEP, 
and perhaps not consistent across rating sites, but as 
indicated, above, no data on inter-rater reliability are 
available for citation. 
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However, it is important to keep in mind that useful general , 
reliability-related inferences can be drawn from observed correl- 
ations among variables. For example, because the level of correla- 
tion between tests is limited by their respective reliabilities, if 
either the shortened SLEP or the writing test has very low relia- 
bility in samples such as those being assessed in the LACCD, the 
scores involved cannot correlate very highly with each other or 
with any other study variable. 

The possibility of systematic differences across colleges with 
respect to "rater bias" is, of course, inherent in any context in- 
volving the generation of ratings in multiple sites, even though a 
common scale is employed, and Butler (1989) called attention to 
anomalous differences, by college, in the average essay ratings of 
students at the same ESL placement level. As noted earlier, such 
average differences by college may reflect substantive differences 
in the average level "writing ability" of the students being asses- 
sed on the respective campuses but systematic differences in rating 
standards may also be involved. SLEP (objective-tost) scores, of 
course, are free of this type of bias. Accordingly, an analysis 
was made of differences between observed essay ratings and 
estimated ratings based on SLEP scores. 

The presence of statistically significant differences 
by college in level of Essay rating relative to SLEP per- 
formance, constitutes a necessary condition for inferring 
differences in rating standards. 

Questions regarding possible effects associated with dif- 
ferences in essay reliability across colleges could not be ad- 
dressed. However, differences by college in all correlational 
findings involving writing samples may to some extent be associated 
with differences in reliability or content (e.g., topics selected), 
and so on . 

The foregoing lines of inquiry were concerned with validity 
properties of the shortened SLEP and the local writing tests. In 
practice, the information provided by the three measures is com- 
bined using nominal weights for percent-right LC, RC, and Essay 
scores. Based on the differences in patterns of nominal weighting, 
noted above, certain patterns of outcomes are expected to obtain: 

In samples from City, Harbor, Mission, Trade, and 
Valley, some two-thirds of the variance in the placement 
composite should be associated with SLEP performance, and 
one-third with essay performance. 

In samples from East and Pierce, on the other hand, 
some 60 percent of composite variance should be associ- 
ated with essay performance, and only 40 percent with 
SLEP performance. At West, as indicated above, for 
samples tested after spring 1991, the composite should be 
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expect ad to reflect differences in essay performance (60 
percent) more than differences in reading ability only 
(40 percent) . 

Regression-based estimates of the relative contribution of the 
test scores to composites were obtained for each college, and 
compared with the expected relative contribution, based on the 
nominal weights. 

Assessing Criterion-Related Validity 

In the LACCD context, all tests used are expected to exhibit 
"reasonable" levels of correlation with appropriate criteria of 
performance in the courses of study to which the testing is 
related — average criterion-related validity coefficients in the 
neighborhood of .35 have been suggested as representing a reason- 
able standard (California Community Colleges, 1992: p. 20). 

Although the foregoing "standard" is proposed, it is specific- 
ally recognized that "the magnitude of the correlation may vary as 
a function of the degree to which a test was used to place stu- 
dents in the course under investigation and/or the variation in 
grading standards across classrooms" (California Community Col- 
leges, 1992: p. 20). Empirical evidence regarding typical levels 
and ranges of test-criterion correlation — either for current 
placement tests or similar tests — in the LACCD context does not 
appear to be available. 

Accordingly, analysis of test/grade correlations in the 
present study was designed primarily to 

(a) generate base-line, empirical findings regarding levels 
and patterns of test/grade correlation for the shortened SLEP and 
the writing test in the LACCD context, and 

(b) assess the extent to which effects associated with use of 
the tests in placement, influence both levels and patterns of 
observed test/grade correlations in ways that limit generalization 
regarding test validity from observed within-course correlations. 



Data and General Study Procedures 

A data file containing over 10,000 records was provided by the 
LACCD District Office. 7 The students involved were assessed for 



The file contained records for an undetermined number of 
students who earned grades in more than one course. These records 
were treated as "unique" for study purposes. However, the number of 
students assessed is, to some extent, less than the number of 
records . 
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placement in Spring 1991, Fall 1991, and Spring 1992, respectively. 
The following data elements were included in the file: 

A. test and demographic data 

1. scores on the placement battery, 

2. the composite score that was used to guide placement, 

3. self-reported background data (birthdate, language, 
educational status, ethnicity) ; 

B. data on student performance, namely, 

1. course grades (grades, either A-F or pass/fail) or 
some other performance-related index (e.g., incomplete, 
withdrawn) ; and 

C. course identification data, including 

1. codes identifying course in which a student was most 
recently enrolled (not necessarily the same as the course in 
which the student was originally placed) , by subject areas 
designated differentially by college: 

a« Developmental Communication , English , and Speech Com - 
munication [hereafter, Speech] at City, East, Harbor, Pierce, 
and Valley) , and 

b. English as a Second Language [ESL 1 through ESL 6] at 
Mission, Trade, and West) . 

Course codes for Mission, Trade, and West (ESL ["400"] 
courses) , directly indicated the proficiency levels associated with 
the correspondingly numbered courses. However, for the other 
colleges this was not the case. 

Level-classif ioation of courses for City, East, Harbor, 
Pierce, and Valley Colleges, respectively, was guided primarily by 
the "English As A Second Language Flow-chart" shown as Exhibit A 
Exhibit A. English As A Second Language Flow chart (City College) 
(from a recent edition of the City College catalog) . Differences 
in ESL curricular-emphasis (grammar, reading/conversation, writing, 
or speaking) are noted in the chart for differently numbered 
courses within each level. 

Section II provides a description of levels and patterns of 
performance on the shortened SLEP and the writing tests in the 
general LACCD population and selected subpopulations; Section III 
provides evidence regarding concurrent- and discriminant-validity 
properties of the tests involved; Section IV reviews findings 
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Exhibit A. English As A Second Language Flow Chart (City College) 
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regarding test/grade correlations; and findings are reviewed and 
evaluated in Section V. 



In presenting and discussing findings and procedures, an 
effort is made to limit the amount of supporting detail considered 
directly in the text; such detail is provided in designated 
appendices . 
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SECTION II. PERFORMANCE OF ESL STUDENTS ON THE SHORTENED SLEP AND 
THE WRITING TESTS 



This section reports the results of analyses that provide an 
overview of performance on the placement tests for ESL students 
generally; also performance of students classified by college and 
by selected demographic characteristics (gender, educational 
status, age, ethnic group, and language) . 

Attention is directed first to descriptive statistics for the 
total sample. Differences in performance on the study variables 
for students attending the respective colleges are then considered. 
Finally, descriptive statistics are provided for ESL students 
classified by gender, educational status, age category, ethnic 
group identify, and language. 



The Total Sample 



Descriptive statistics are shown in Table 2.1 for designated 
study variables, based on data for total LACCD sample. Means and 
standard deviations, and minimum and maximum values, shown for each 
variable, are based on the total number of cases with observations 
on the corresponding variable. In some situations, especially at 
West, only the shortened SLEP reading section was administered. It 
can be seen that some 6,500 (about 60 percent) of the originally 
assessed students were sufficiently "persistent" in the ESL pro- 
gram, to earn a grade (on either the A-F or Pass/ Fail scales) in an 
ESL course. 

Average percent-right scores for the SLEP are shown to provide 
a basis for inferences regarding the difficulty level of the 
respective sections and the combined score for the sample as a 
whole. The mean percent right score for the essay is shown for 
perspective only — inferences from percent-right scores regarding 
"appropriate level of difficulty" for the essay cannot be thought 
of as comparable to corresponding inferences from percent-right 
score on a standardized test such as the shortened SLEP. 

The average percent right score of approximately 68 for 
the shortened LC score indicates that this section is of 
about "middle difficulty" for the sample, and the cor- 
responding mean of about 60 percent for the RC section 
indicates that the section is of somewhat greater than 
middle difficulty for the sample. The total shortened 
SLEP mean (LC + RC) of 58 translates directly into a 
percent-right equivalent — indicating that the items 
selected for the shortened SLEP are generally just below 
middle difficulty (approximately 67 percent for four- 
^option, rights-scored multiple-choice tests) , a diffi- 
culty level that is psychometrically efficient for 
measurement purposes. 
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Table 2.1. Descriptive Statistics for Study Variables, All LACCD 
(Spring 1991 through Spring 1992) 



Variable* 










Valid 


Mean 


Std Dev 


Minimum 


Maximum 


N 


LC 


30.45 


9.88 


1 


45 


9935 


RC , 


27.24 


10.75 


1 


55 


10695 


SLEP 


57.96 


18.98 


3 


100 


9858 


ESSAY 


3.19 


1.46 


0 


8 


10786 


LCPCT 


67.66 


21.36 


2.2 


100 


9935 


RCPCT 


49.54 


19.55 


1.8 


100 


10695 


ESSPCT 


39.83 


18.20 


.0 


100 


10786 


GRADEAF 


2.63 


1.19 


0 


4 


5055 


PASS FAIL 


.68 


.47 


0 


1 


1526 


AGE 


29.87 


9.01 


10. 3 


83. 5 


10789 



Note. See Appendix B.l for full distributions of LC, RC, 
SLEP (shortened version), and Essay scores; see 
Appendix B.2, for descriptives statistics by college. 

* LC Listening Comprehension, short SLEP 

RC Reading Comprehension, short SLEP 

SLEP LC + RC, short SLEP (100 items) 

Essay Writing sample 

LCPCT Percent- right conversion of LC 

RCPCT Percent- right conversion of RC 

ESSPCT Percent- right conversion of Essay 

GRADEAF Grade on the A to F scale (4,0) 

PASSFAIL Grade on the pass/fail (1,0) scale 
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A similar conclusion was reached by Butler (1989: p. 25), based on 
evidence indicating that none of the independently defined 
prof icency-level subgroups studied ..." 'topped out' or 
•bottomed out 1 " (p. 25) on either the listening comprehension or 
the reading comprehension sections as a whole, or on the respective 
item- type subsections. Full distributions if scores on the 
shortened SLEP sections and the writing sample are shown in 
Appendix B.l. 

Differences By College 

Table 2.2 shows means and standard deviations of distributions 
of scores on the shortened SLEP and the Essay rating, respectively , 
by college. Differences in average performance are portrayed 
graphically in Figure 2.1. Note that mean Essay rating has been 
multiplied by 10, and that the eight colleges are arrayed in 
descending order (left to right) with respect to mean score on the 
shortened SLEP (LC+RC) . 

Butler (1989: p. 18-19) noted that there were differences by 
college in average essay rating for students at the same., independ- 
ently defined levels of proficiency. It can be seen in Figure 2.1, 
that 

(a) differences among colleges with respect to mean Essay 
rating are much more pronounced than are differences with respect 
to mean SLEP performance, and 

(b) mean essay rating does not vary systematically with mean 
SLEP score across colleges — for example, three of the four highest 
means for the Essay are registered by the four colleges with the 
lowest-scoring students on the shortened SLEP. 

As will be seen in Section III, scores on the shortened SLEP 
and Essay rating covary quite systematically within the respective 
colleges. Lack of corresponding covariation among the college 
means on these variables, suggests that raters at different 
colleges tend to have different "rating standards." Such dif- 
ferences introduce interpretive complications in analyses of 
average performance on the Essay relative to performance on the 
SLEP when data are aggregated across colleges. 

Performance by Demographic Categories 

In the study (Butler, 1989) that was conducted as part of the 
process through which the SLEP was selected as the standardized- 
test component of the LACCD placement battery, demographic 
characteristics were not considered. Butler noted, simply (p. 5) 
that "... many of the students who enroll in LACCD classes have 
recently come from high schools . . . . " Table 2 . 2 and Figure 2 . 1 
show differences by college in performance on the basic study 
variables . 
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Table 2.2. Sunnary Statistics By College 



Shortened SLEP 

College Code M LC RC SLEP ESSAY 









H 


Sd 


n 


Sd 


n 


Sd 


H 


Sd 


City 


1 


3872 


27 


10 


24 


10 


51 


19 


3.2 


1.5 


East 


2 


942 


35 


7 


31 


10 


65 


15 


2.5 


.9 


Harbor 


3 


431 


35 


7 


33 


9 


68 


15 


2.9 


1.2 


Mission 


4 


1717 


27 


11 


28 


11 


56 


20 


2.7 


1.4 


Pierce 


5 


493 


39 


5 


38 


9 


77 


13 


4.7 


1.3 


Trade 


6 


477 


30 


7 


23 


8 


53 


13 


4.5 


1.4 


Valley 


7 


1616 


35 


7 


30 


10 


65 


15 


2.9 


1.3 


West 


8 


1241 


32 


9 


27 


10 


59 


18 


3.7 


1.2 


Total 




9855 


31 


10 


27 


11 


58 


19 


3.1 


1.5 



Figure 
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2.1. Performance of students on placement variables, 
by college 

Measure 

» SLEP LC (S) 

' o SLEP RC (S) 

□ Essay Rating 

■ ■■ * • % 

/ \ (x 10) 




Note. Colleges listed in descending order (left to right) 
with respect to mean total score on shortened SLEP. 



Pierce East West Trade 

Harbor Valley Mission City 
College 
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Data shown in Tables 2.3 through 2.7 indicate that the ESL 
population being assessed with the shortened SLEP is extremely 
heterogeneous with respect to age, educational level, ethnicity, 
and language background. Mean scores on the shortened SLEP and the 
writing sample are provided for subgroups classified by gender 
(Table 2.3) , educational status (Table 2.4) , age group (Table 2.5) , 
ethnic group (Table 2.6) and language background (Table 2.7). 

Figure 2.2 highlights diversity in educational level, and 
diversity in language background. 

o The majority of students hold secondary-school diplomas 
earned in schools located outside the United States, but 
the sample includes individuals who have not completed 
secondary school — some enrolled in adult education 
courses, some as special students — as well as individuals 
with bachelor's or higher degrees. 

o The largest linguistic subpopulation is made up of 
native-speakers of Spanish (about 44 percent of the 
total) ; none of the other directly identified language 
groups accounts for much more than 10 percent (11 percent 
reported Armenian, and less than 1 percent reported 
Filipino) . 

Lack of systematic covariation across colleges between means 
for the writing sample and means for the SLEP has already been 
noted (see Figure 2.1 and related discussion, above), suggesting 
that site-differences in rating standards may be involved. Depite 
such local differences, Figure 2.3 shows a systematic pattern of 
covariation between shortened SLEP means and Essay means across 
subgroups classified by educational level (see Table 2.3 ^lor 
detail) . 

• The subgroups are arrayed (left to right) in descending 
order with respect to mean score on the shortened SLEP. 
Note that the mean Essay rating has been multiplied 
(arbitrarily) by 20. The pattern shown in Figure 2.3 
indicates that average level of performance on the Essay 
tends to decrease as average level of SLEP performance 
decreases. 

• The consistent relationship indicated in Figure 2.3 
suggests that subgroup members at "higher-rating" 
colleges were balanced by members of the same subgroups 
at "lower-rating" colleges, hence the theoretically 
consistent set of findings for the variables involved in 
a general sample despite local differences in rating 
standards. Generally similar trends (positive covar- 
iation between SLEP mean and Essay mean) can be discerned 
by inspecting the means reported in the various tables. 
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Table 2.3. Suwaary Statistics by Gender 



Shortened SLEP 



Gender 


Code N 


LC 


RC 


SLEP 


ESSAY 






M Sd 


M Sd 


M 


Sd 


M Sd 


Male 


1 4250 


32 9 


29 11 


61 


18 


3.3 1.4 


Female 


2 5605 


29 10 


26 11 


55 


19 


3.1 1.5 


Total 


9855 


31 10 


27 11 


58 


19 


3.1 1. 



Table 2.4. Suwaary Statistics By Reported Educational Status 

Shortened SLEP 



Educational Code N LC RC SLEP ESSAY 



status 






M 


Sd 


M 


Sd 


M 


Sd 


M 


Sd 


Missing 


0 


1061 


27 


11 


26 


11 


54 


20 


2.8 


1.4 


Special student 


1 


805 


34 


9 


31 


11 


65 


18 


3.5 


1.5 


Adult schooK HS 


2 


134 


28 


10 


24 


10 


52 


18 


2.9 


1.5 


Adult school < HS 


3 


926 


27 


10 


24 


10 


51 


17 


2.9 


1.4 


Unenrolled < HS 


4 


846 


29 


11 


26 


11 


55 


20 


2.9 


1.5 


GED/HS equiv 


5 


259 


33 


9 


32 


11 


65 


18 


3.5 


1.4 


CA HS certificate 


6 


31 


33 


11 


33 


11 


67 


21 


3.4 


1.4 


Foreign HS dipl 


7 5804 


31 


10 


27 


10 


57 


18 


3.2 


1.4 


Associate degres 


8 


194 


34 


9 


31 


10 


65 


18 


3.6 


1.4 


Bachelor's/plus 


9 


729 


34 


8 


33 


10 


68 


16 


3.7 


1.4 


Total 




9855 


31 


10 


27 


11 


58 


19 


3.1 


1. 



Table 2.5. Suntnary Statistics by Age Group 

Shortened SLEP 

Age group Code N LC RC SLEP ESSAY 







M 


Sd 


M 


Sd 


M 


Sd 


M 


Sd 


Below 22.5 yrs 


2187 


34 


8 


30 


10 


64 


16 


3.4 


1.4 


22.5 - 26.4 yrs 


1922 


32 


9 


28 


10 


60 


18 


3.2 


1.5 


26.5 - 30.4 yrs 


1780 


31 


10 


27 


11 


58 


19 


3.2 


1.5 


30.5 - 36.4 yrs 


196S 


30 


10 


26 


11 


56 


19 


3.0 


1.4 


36.5 yrs & older 


1998 


27 


11 


25 


12 


52 


20 


2.9 


1.5 


Total 


9855 


31 


10 


27 


11 


58 


19 


3.1 


1.5 
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Table 2.6. Sunoary Statistics by Reported Ethnic Group Membership 



Shortened SLEP 

Ethnic Code N LC RC SLEP ESSAY 



group 






M 


Sd 


M 


Sd 


K 


Sd 




M 


Sd 


Missing 


0 


209 


31 


10 


29 


11 


61 


19 


2. 


8 


1 .5 


Chinese 


10 


635 


35 


7 


31 


9 


66 


15 


3! 


1 




Japanese 


11 


617 


37 


5 


32 


7 


69 


11 


3. 


8 


1.2 


Korean 


12 


869 


34 


6 


30 


8 


64 


13 


3. 


6 


1.4 


Laotian 


13 


31 


25 


9 


23 


7 


48 


15 


2. 


7 


1.4 


Cambodian 


14 


15 


26 


12 


26 


11 


51 


22 


3. 


5 


1.8 


Vietnamese 


15 


321 


31 


8 


28 


10 


59 


16 


3. 


4 


1.5 


Indian(Sub) 


16 


70 


32 


10 


31 


13 


63 


21 


3. 


8 


1.4 


Other Asian 


19 


697 


32 


9 


27 


10 


59 


17 


3. 


2 


1.4 


Black 


20 


124 


28 


7 


25 


8 


52 


14 


3. 


6 


1.4 


Fi I i pi no 


30 


91 


39 


4 


39 


7 


78 


10 


4. 


0 


1.4 


Mexi can* 


AO 


2022 


29 


10 


28 


11 


57 


19 


2. 


9 


1.5 


Central Amer 


41 


1599 


26 


11 


25 


11 


54 


20 


3. 


2 


1.5 


South Amer 


42 


654 


31 


10 


28 


11 


59 


20 


3. 


3 


1.4 


Other Hispanic 


49 


818 


28 


11 


26 


11 


55 


20 


3. 


2 


1.5 


Caucasian 


50 


1758 


28 


11 


23 


11 


51 


20 


3 


0 


1.5 


American Indian 


60 


3 


40 


1 


27 


9 


66 


10 


3 


7 


.6 


Other Pacific 


79 


56 


38 


5 


33 


10 


71 


14 


4 


1 


1.4 


Other Non-Uhite 


80 


87 


35 


9 


26 


10 


62 


18 


3 


3 


1.3 


Decline 


90 


113 


35 


8 


30 


11 


65 


18 


3 


4 


1.5 


Total 




9855 


31 


10 


27 


11 


58 


19 


3.1 


1. 



Table 2.7. Summary Statistics by Language 

Shortened SLEP 

Language Code N LC RC SLEP ESSAY 









M 


Sd 


M 


Sd 


M 


Sd 


M 


Sd 


Missing 


0 


607 


32 


9 


29 


11 


61 


19 


3.3 


1.6 


Engl :sh 


1 


192 


33 


9 


31 


11 


65 


18 


3.3 


1.5 


Armenian 


2 


1216 


25 


10 


19 


9 


44 


17 


2.6 


1.4 


Chinese 


3 


607 


34 


7 


31 


9 


65 


15 


3.1 


1.3 


Farsi 


4 


252 


36 


7 


28 


10 


64 


15 


3.3 


1.3 


Fi lipino 


5 


80 


38 


6 


38 


8 


76 


11 


3.9 


1.4 


Japanese 


6 


604 


37 


5 


32 


7 


69 


10 


3.8 


1.2 


Korean 


7 


858 


34 


6 


30 


8 


64 


13 


3.6 


1.4 


Russian 


8 


354 


32 


9 


26 


10 


58 


17 


3.3 


1.4 


Spanish 


9 


4722 


29 


10 


27 


11 


56 


20 


3.1 


1.5 


Vietnamese 


10 


297 


31 


8 


28 


10 


59 


15 


3.4 


1.5 


Other 


11 


1000 


35 


8 


30 


11 


66 


17 


3.6 


1.5 


Total 




9855 


31 


10 


27 


11 


58 


19 


3.1 


1.5 
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Figure 2.2. Distribution of sample by educational 
status and by language background 
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Figure 2.3. Trends in means on the Essay and the shortened 
SLEP. respectively, by educational status 
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The findings summarized above indicate that 



(a) LACCD colleges are serving an ESL student population that 
is remarkably heterogeneous with respect to age, educational 
status, ethnicity, and native language, 

(b) the shortened SLEP sections appear to be at psychometric- 
ally efficient levels of difficulty for the general LACCD ESL 
population as well as for subgroups that differ markedly with 
respect to age, educational level, language background, and self- 
reported ethnic group membership, and 

(c) subgroup means on the shortened SLEP and the Essay, 
respectively, exhibit a systematic, positive relationship — true for 
subgroups classified by educational level, and for other 
demographic subgroups as well. 

Related Findings 

Some related findings are provided for additional perspective 
in Appendix B.2 (test performance by ESL placement level and by 
college), Appendix B.3 (test performance by language group and by 
college) , and Appendix B.4 (average differences in test performance 
by gender, age, and educational level, by college) . 

Regarding the latter, it is useful to note the general pattern 
of findings: 

• females tend to earn lower scores on the placement 
variables than do males, 

• older students tend to earn lower scores than do 
younger students, and 

• students with less formal education tend to earn lower 
scores than do students with more formal education. 8 

The findings summarized in this section suggest that both the 
shortened SLEP and the local writing tests are providing valid 
information regarding individual and group differences in the LACCD 
context, a conclusion that is reinforced by findings in subsequent 
sections of this report. 



It is noteworthy that the general negative relationship 
between age and test performance is stronger for listening 
comprehension than for either reading comprehension or writing 
ability (essay rating) . Pursuit of these and other theoretically 
and pragmatically interesting questions is beyond the scope of the 
present study. 
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SECTION III. CONCURRENT- AND DISCRIMINANT-VALIDITY PROPERTIES OF 
THE SHORTENED SLEP AND THE WRITING TESTS 

SLEP listening comprehension items have face validity as 
measures of the ability to comprehend utterances in English, and 
SLEP reading comprehension items are face valid measures of the 
ability to comprehend the meaning of material that is written in 
English (see Appendix A. 3). Writing samples have general face 
validity as measures of ability to write comprehensibly using 
English. 

Based on previous research, moderate to relatively strong 
levels of correlation have been found to obtain among both direct 
and indirect measures of the four basic language macroskills — 
including speaking, not here under consideration — in samples of 
educated ESL users/learners (e.g., Pike, 1979; Oiler, 1983: passim; 
Hale, 1986; Wilson, 1989), and in samples studying languages other 
than English, as foreign languages (see especially, Carroll, 1967). 

In a comprehensive study (Carlson, Bridgeman, Camp, and Waan- 
ders, 1985) associated with development of the Test of Written En- 
glish (TWE) , holistic scores for ESL-writing samples were found to 
be somewhat more closely related to scores on the nonlistening por- 
tions of the TOEFL (Structure and Written Expression [SWE] and 
Reading Comprehension and Vocabulary [RCV] ) , respectively, than to 
the TOEFL Listening Comprehension score. 

With some variation in detail (by essay topic) the SWE 
and RCV scores correlated approximately .6 with essay 
rating while TOEFL LC scores correlated at approximately 
the .5 level. Inter-rater reliabilities centering around 
.70 were reported (as estimates of reliability " ... 
if only the scores from one judge are to be used 
operationally" (p. 57) . 

The SLEP Manual (e.g., ETS, 1991) reports evidence indicating 
that scores on the SLEP test are relatively strongly related to 
scores on the TOEFL. 

More specifically, SLEP Listening Comprehension (LC) 
score correlated more highly with TOEFL LC score (.74) 
than with either the TOEFL Structure and Written 
Expression (SWE) score or the TOEFL Vocabulary and 
Reading Comprehension (RC) score; SLEP Reading 
Comprehension (RC) score correlated equally highly with 
TOEFL LC (r = .80) and TOEFL RC (r = .79); the 
correlation between the two total scores in this sample 
was .82, slightly lower than that between SLEP RC and 
TOEFL Total (.85). These relationships obtained in 
samples of ESL students in intensive ESL programs on four 
U. S. college campuses. 
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Because of the strong relationship between the two tests, it 
is a reasonable inference that the SLEP test shares to some extent 
TOEFL 's validity-related properties — properties that have been 
extensively investigated (see, for example, ETS, 1992a) . The 
findings reported below support such an inference. 



Findings Regarding Concurrent and Discriminant Validity 

Concurrent correlations of scores on the shortened SLEP test 
with Essay scores, computed by college, are shown in lable 3.1. 
Standard deviations of scores on the shortened SLEP test and the 
writing samples involved are also shown. Several aspects of the 
findings are noteworthy: 

(1) Concurrent relationships were moderate to strong in 
each setting. 

(2) The RC/Essay relationship typically was stronger than 
the LC/Essay relationship, consistent with expectation— 
that is, reading and writing skills should tend to be 
more close-ly related than listening and writing skills. 
At the same time, coefficients for total score on the 
shortened SLEP test (LC + RC) were somewhat larger than 
those for RC alone, indicating that the items in the 
shortened LC section are providing some unique 
information regarding aspects of the ability being 
measured by the writing test. 9 

(3) The pooled, wi thin-school coefficients typically were 
larger than the "total sample" coefficients, the latter 
being attenuated by lack of systematic covariation in the 
corresponding means across schools — assumed to reflect 
non-validity-related differences in "rating standards." 



Concurrent relationships were stronger and the patterning of 
relationships — RC/Essay coefficients higher than LC/Essay 
coefficients — was more consistent with expectation, in samples 
assessed at City, Mission, West, and Pierce, than in samples 
assessed at East, Harbor, Valley or Trade. 



It is worth noting in this connection that Dictation items 
included in the shortened LC section are highly integrative, in 
that in order to identify the written sentence that matches the 
spoken stimulus, examinees must read four sentences. See illus- 
trative items in Appendix A. 3. This particular issue is beyond the 
scope of the present study, but is a fruitful one for further 
investigation . 

LACCD/SLEP Study 26 



46 



Table 3.1. Concurrent Correlations of Scores on the Shortened 
SLEP with Essay Score, By College 



College 



N 



Correlation with 
essay 

LC RC LC+RC 



City 

East 

Harbor 

Mission 

Pierce 

Trade 

Valley 

West 

TOTAL 

WITHIN* 



3,323 
951 
514 

1,965 
511 
648 

1, 641 

1,271 
10,590 



,64 
.45 
.49 
.56 
.48 
.40 
.43 
.53 
.49 
.55 



.67 
.51 
.48 
.58 
.57 
.37 
.47 
.57 
.49 
.57 



.70 
.53 
.51 
.60 
.58 
.44 
.51 
.59 
.53 
.61 



SLEP 
r 



Standard deviation** 
LC RC SLLP Essay 



Mission 

City 

West 

Valley 

East 

Harbor 

Pierce 

Trade 



.59 
.70 
.59 
.51 
.53 
.51 
.58 
.44 



11 
10 
9 
7 
7 
7 
5 
7 



11 
10 
10 
10 
10 
9 
9 
8 



20 
19 
18 
15 
15 
15 
13 
13 



1.4 
1.5 
1.2 
1.3 
.9 
1.2 
1.3 
1.4 



TOTAL 



10 



11 



19 



1.5 



Note. These are "missing data" correlations: each coef- 
ficient reported is based on data for all cases with 
observations on the pair of variables involved. 

* The coefficients in this row are, in effect, size- 
adjusted means of the coefficients for the schools. 

** Colleges are listed in descending order with respect 
to standard deviation of scores of the shortened SLEP 
test (LC + RC) . 
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Three of the four highest (lowest) SLEP/Essay coef- 
ficients were for samples from one of the four highest- 
ranking (low st-ranking) colleges with respect to SLEP 
standard deviation. 

The general pattern of differences by college in levels of 
concurrent relationship is discernible in the results of analyses 
conducted by college for Spanish-speaking and Korean-speaking 
subgroups, respectively, reported in Table 3.2. 

By and large, the levels of concurrent relationships 
observed in general samples by college tend to be present 
in college-level samples for the two language groups 
involved. 

The findings reported above attest to the validity of both the 
shortened SLEP and the local writing tests. 

It is particularly noteworthy in this connection that the lev- 
els of concurrent relationship observed in the LACCD context, be- 
tween scores on a shortened version of the SLEP and scores on local 
writing tests — with attendant differences in topic, rating pro- 
cedures, and so 6n — parallel levels that have been found to obtain 
between scores on the Test of English as a Foreign Language and the 
Test of Written Expression (writing samples elicited under standard 
conditions, scored under controlled conditions by at least two 
raters, and so on) in (more highly selected) 

(a) samples tested in developmental research (Carlson, 
Bridgeman, Camp, and Waanders, 1985) involving the TOEFL and pro- 
totypical versions of the TWE, and 

(b) samples taking both tests under fully operational con- 
ditions (see ETS, 1992b) . 10 

Such findings attest to the validity and relevance for place- 
ment of both the SLEP and the local writing tests. The general is- 
sue of differences across colleqes in "rating standards" is ad- 
dressed in the following section". 



1U TOEFL/TWE correlations have been reported (e.g., ETS, 1992b: 
p. 13) , by world region, for large, relatively highly selected ESL- 
samples taking both tests in operational administrations. Weight- 
ed averages of regional TOEFL/TWE correlations (corrected for un- 
reliability of TOEFL scores) , for almost 90,000 examinees tested in 
May 1991, are as follows: TOEFL Listening Comprehension (.59), 
TOEFL Structure and Written Expression (.61), TOEFL Reading 
Comprehension and Vocabulary (.59), and TOEFL Total (.63). 
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Table 3.2. Concurrent Correlations of Shortened SLEP Scores with 
Essay Rating for Two Language Groups, By College 



Correlation with Essay rating 
Shortened SLEP Test 



T.ancmacfe 


N 




RC 


SLEP 


/College 










Spanish 










City 


1, 416 


. 68 


.70 


. 73 


Mission 


1 299 


. 51 


54 


. 57 


Pierce 


77 


. 59 


. 62 


. 66 


West 


146 


.62 


.61 


. 65 


East 


327 


. 49 


. 53 


. 55 


Harbor 


1 28 


.44 


. 32 


. 41 


Trade 


322 


. 29 


. 27 


. 32 


Valley 


554 


.46 


.49 


.53 


Korean 










City 


570 


.63 


.69 


.74 


Mission 


21 


.57 


.65 


.68 


Pierce 


39 


.70 


.69 


.73 


West 


36 


.56 


.54 


. 62 


East 


15 


.44 


.78 


.75 


Harbor 


15 


.48 


.36 


.41 


Trade 


24 


.54 


.30 


.43 


Valley 


98 


.42 


.55 


.57 



Note. Concurrent correlations tend to be higher in general 
samples at City, Mission, Pierce, and West, than at the other 
four colleges. 
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Differences in Rating Standards by College 



Explication of the differences in levels and patterns of 
concurrent- and discriminant -validity by college is beyond the 
scope of this inquiry — such differences plausibly are due in part 
to effects associated with differences in dispersion by college 
inSLEP scores, differences in the reliability of ratings, and other 
factors. Data needed to pursue these and related issues directly 
are not available. However, the analyses reported below shed 
further light on the question of systematic differences by college 
in "rating standards," suggested by the lack of systematic 
covariation across colleges between mean writing score and mean 
score on the shortened SLEP. 

A systematic assessment of differences in Essay performance 
relative to shortened SLEP performance was made, by analyzing 
average discrepancies between observed Essay rating and Essay 
rating estimated from SLEP scores, using a regression equation 
derived for the total sample (see Appendix B.5 for detailed results 
of the regression analysis) . Mean residuals were analyzed by 
college, and for subgroups by gender, language, educational level, 
and age, respectively. 

Salient results of these analyses are summarized in Figures 
3.1 (college differences in mean residuals) and Figure 3.2 (other 
subgroup differences, plus college differences from Figure 3.1 for 
"same-scale" perspective) . Positive discrepancies (residuals) 
indicate high Essay ratings relative to SLEP scores, and negative 
residuals indicate the opposite. 

It is apparent that differences in mean residuals are much 
greater in the analysis by college, than in analyses involving the 
respective subgroups (for which data are aggregated across col- 
leges) . Of course, the observed mean residuals for the demographic 
subgroups may to some extent be artifactually inflated or deflated 
by college-level effects — for example, differential concentration 
of subgroups in higher- or lower-rating colleges. In the circum- 
stances, interpretive inferences need to be drawn with caution. 11 



In view of apparent differences across sites in "rating 
standards," pending resolution of questions regarding comparability 
of Essay scores across colleges, future investigations of subgroup 
differences in the relative development of essay-assessed "writing 
ability" and SLEP-assessed skills, need to be conducted at the 
college level. These and related issues are beyond the scope of 
the present study, but they constitute meaningful topics for 
further inquiry. More generally, the findings suggest the need for 
systematic study of the rating process, including the collection of 
data needed to evaluate degree of agreement among raters with 
respect to level as well as rank-order. 
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"gure 3.1. Mean residuals (Essay - Essay estimatea 
from shortened SLEP scores) by college 
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Figure 3.2. Mean residuals for designated subgroups 
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With these limitations in mind, it is of interest to note that 
mean residuals were larger in analyses by college than in the other 
analyses. Generally speaking, 

• mean residuals by gender and by age group were extremely 
small ; 

• the pattern of residuals by educational level indicates 
simply that more highly educated students tend to earn 
somewhat higher Essay ratings, on the average, than 
expected from their shortened SLEP scores, and that the 
opposite was true for less highly educated students; and 

• Chinese-speakers and Korean-speakers, albeit to a 
lesser extent, earned somewhat higher than expected Essay 
scores . 

Again, these analyses do not take into account possible 
differences in concentration of subgroups by college. 

Related Analyses: Weighting of Tests in the Placement Composite 

In practice, shortened SLEP scores and Essay ratings are 
combined for placement purposes. As indicated earlier, raw scores 
on the shortened LC and RC sections of the SLEP test and the 
writing sample, respectively, are converted to "percent right" 
equivalents, and then combined using predetermined nominal weights, 
as indicated below: 

Nominal weight 
College Essay LC RC 

City, Harbor, Mission, 

Trade, Valley, West 12 .34 .33 .33 

East, Pierce .60 .20 .20 



Based on these nominal weights, the placement of students at 
the first group of colleges is expected to be influenced more by 
their SLEP scores than by the essay rating, while the opposite is 
true for the placement of students at the latter two colleges. 
However, the actual contribution of variables in a composite 
(weighted sum of scores on the variables) may differ from that 
indicated by the nominal weights involved (e.g., Guilford, 1965: 
pp. 424-425). 



The pattern indicated here is applicable only to about one- 
third of the sample from West. only the Essay rating and the 
shortened SLEP test reading score are currently used, weighted 
60/40. For present purposes, only the sample with the "34/33/33" 
composite is under consideration. 
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The actual composites computed on the respective campuses were 
included in the LACCD file. To assess the actual relative weight- 
ing of the three test scores in these composites, analyses were 
made of the "structure" of the composites used on the respective 
campuses — that is, the composite for a given campus was regressed 
on the three scores involved, of interest are findings regarding 
the standard partial regression (beta) weights for the tests. 

Salient findings are summarized in Table 3.3. 

• Scores on the shortened SLEP test contributed roughly 
two-thirds of the variance when nominal wrights of 
". 3 4/. 3 3/. 3 3" were employed; Essay scores contributed 
more than did SLEP scores when nominal " .60/. 20/. 20 11 
weights (or the variation employed st West) were used. 

It is apparent that the actual weighting of the tests conforms 
relatively closely to the nominally specified weighting, with some 
variation in detail from college to college. Knowledge of the 
relative contribution of the variables in the placement composites 
used is important for evaluating observed correlations between 
course grade and scores on the placement variables. 

More specifically, for example, restriction of range effects 
are likely to be more pronounced for SLEP scores than for Essay 
scores when placement is guided by the ".34/. 33/. 33" pattern of 
weights, and the opposite is likely to be true for situations in 
which the composite reflects relatively more Essay variance than 
SLEP variance. More direct evidence at the degree of curtailment 
due to use of tests in placement, and the attendant distorting 
effects on observed correlations is provided below. 

Range Restriction in Placed Samples 

One consequence of using test scores to place students is that 
the range of talent within the resulting samples is automatically 
restricted. The extent to which placement score distributions are 
restricted in placed samples is suggested in Figure 3.3, which 
shows distributions of standardized scores on a linear composite of 
percent-right scores on shortened SLEP LC and RC, and Essay, 
respectively > weighted to reflect their contribution to differences 
among the seven placement-level subgroups. 13 



The method of multiple discriminant analysis (e.g., SPSS, 
1990: p. 127-140) was used to assess differences among the seven 
groups with respect to the three test variables. One linear 
discriminant function accounted for 98 percent of observed 
differences among these groups; the percentage contribution of the 
respective tests was estimated as 35-35-32 for Essay, RC, and LC, 
respectively, consistent with the regression outcomes reported 
above . 
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Table 3.3. Effective Contribution of Components in the LACCD 
Placement Battery, By Institution 



Ksta weights Percentage contribution 



College* 


N 


LC 


RC 


Es- 


LC 


RC 


Es- 


SLLP 


Es- 










say 






say 




say 


City 


3 f 833 


.43 


.35 


.35 


38 


31 


31 


69 


31 


Harbor 


431 


.40 


.43 


.41 


32 


35 


33 


67 


33 


Mission 


1, 642 


.46 


.42 


.27 


40 


37 


23 


77 


37 


Trade 


474 


.35 


.41 


.59 


26 


31 


43 


57 


43 


Valley 


1, 616 


.38 


.46 


.41 


30 


37 


33 


67 


33 


West (a) 


380 


.40 


.39 


.37 


34 


34 


32 


68. 


32 


East 


941 


.27 


.30 


.62 


23 


26 


52 


49 


52 


Pierce 


490 


.19 


.25 


.71 


16 


22 


62 


38 


£2 


West(b) 


858 




.53 


.61 




46 


54 


46 


M 



Note ; The composite reported by each school was regressed on LC, 
RC, and Essay. The "beta weights" are the resulting standard 
partial regression weights. Percentage contribution is the percent 
conversion of the ratios of the respective beta weights to the sum 
of all three weights. 

* Colleges are grouped by known differences in formulae for 
weighting the placement variables. The first five colleges used 
the standard weighting (.33*LC + ,33*RC + .34*Essay). East and 
Pierce used the 60/20/20 pattern. West(b) represents use of RC and 
Essay only, weighted .40(RC) and .60(Essay); West(a) reflects use 
of the full battery, with standard weighting (Spring 1991 only) . 
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Figure 3.3. Relative frequency distributions for a 
composite of LCpct, RCpct. and ESSAYpct scores, weighted 
according to contribution to discrimination of subgroups 
classifed by ESL levels at City: Levels 1 thru 6 only 
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The foregoing analyses were based on data for City College 
only, and distributions are shown for students in the first six ESL 
placement levels only. u The distributions in Figure 3.3 overlap 
to some extent, indicating that these level classifications reflect 
factors in addition to test performance: initial consideration of 
background factors, shifts in course assignments, and so on. How- 
ever, for present purposes, the important point is that use of the 
tests in placement generates course-level samples that are rela- 
tively homogeneous with respect to measured proficieny, with at- 
tendant distorting effects on observed correlations among the 
variables involved. 

Such effects are clearly illustrated in Table 3.4, which shows 
intercorrelations of test scores in a restricted (English 85) at 
City College, and the corresponding intercorrelations in the total 
City sample. It is evident that the correlations observed within 
the restricted sample bear little resemblance to those observed in 
the unrestricted (preplaced) general sample. 

Implications for assessing criterion-related validity . By 
extension, observed correlations between scores on the placement 
battery, on the one hand, and course grade, on the other, in re- 
stricted course-level samples (e.g. , students taking English 85) 
are likely to be similarly limited as bases for inferences 
regarding the validity of the tests as measures of aspects of 
English proficiency in the ESL population under consideration. 

Limitations of test/criterion correlations in highly selected 
samples were clearly illustrated by the findings of a classic em- 
pirical study reported by Thorndike (1949: p. 170-171) , in which 
aspirants for pilot training were admitted without regard to their 
performance on the selection tests . After criterion data became 
available for all members of the unrestricted, experimental 
sample, validity coefficients were computed for (a) the unre- 
stricted experimental group and (b) a restricted subgroup made up 
only of individuals who would have qualified based on their test 
performance. Validity coefficients for the various selection tests 
in the unrestricted and restricted samples, respectively, were as 
shown in Table 3.5. 

o Coefficients for the "restricted" group clearly are 
lower than those for the "unrestricted" group. 



Students in Level 7 courses (largely Speech Communication) 
had lower average scores on the placement composite than did 
students at Level 6. This was true at City and at the other 
colleges as well (see Appendix B.2, for example). Reasons for this 
are not clear. In any event, data for the first six levels are 
sufficient to make the point at issue here. 
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Table 3.4. Intercorrelations of Placement Variables in Restricted 
and Unrestricted Samples: Illustrative Data, City college 



Variable ESSAY 


LC 


RC 


SLEP 


PlcCom] 


Restricted 










English 85 (N = 269) 










Mean 4 . 6 


36.1 


33.6 


69.7 


66. 1 


Sd . 8 


4.0 


5.3 


6.4 


4.8 


ESSAY 1.0000 


-.18 


-.11 


-.20 


.5? 


LC .62 


1.00 


-.08 


.57 


.44 


RC .66 


.73 


1.00 


.78 


.54 


SLEP .69 


.93 


.93 


1.00 




PlcComposite .85 


.90 


-sJ9_ 


.97 


1.00 


Mean 3 . 2 


27.0 


23.6 


50.6 


47.7 


Sd 1.5 


10.1 


10.2 


18.9 


17. 6 



All students (N = 2,389) 
Unrestricted 



Note. Coefficients above the diagonal are for the English 85 
sample, those below the diagonal are for the total sample. 
Underlining indicates spuriously high correlation (part-whole 
relationship) . 



Table 3.5. Illustrative Effects of Selection on Observed Validity 
Coefficients (from Thorndike, 1949: p. 171) 

Validity coefficients 

Total group* Qualified group** 



(N = 


1,036) 


(N = 136) 


Composite 


.64 


. 18 


Mechanical Principles Test 


.44 


.03 


General Information Test 


.46 


.20 


Complex Coordination Test 


.40 


-.03 


Instrument Comprehension Test 


.45 


.27 


Arithmetic Reasoning Test 


.27 


.18 


Finger Dexterity Test 


.18 


. 00 



* Tested but admitted without regard to test score 
(unrestricted) . 

** Only those who would have been accepted if test scores 
had been considered (restricted) . 



LACCD/SLEP Study 37 

57 



9 

ERIC 



o The relative size of the coefficients in the restricted 
group does not provide a basis for inferences regarding 
the relative validity of the tests when used with the 
entire applicant population. 

As will be seen in the following section, both levels and 
patterns of test/grade correlations reflect range-restriction 
effects such as those considered illustratively above. 
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SECTION IV. OBSERVED CORRELATIONS OF PLACEMENT TESTS WITH COURSE 
GRADE 

Findings reviewed in the preceding section indicate that 
within-college correlations between scores on the locally developed 
writing tests and scores on the shortened SLEP tend to center at 
about .60, varying between roughly .5 and .7 across colleges. 
These correlations were in general (pre-placement) samples from the 
LACCD population of ESL students. 

• The findings constitute novel empirical evidence of the 
validity of the locally developed writing tests; they 
attest directly to the validity of the shortened SLEP, 
and logically extend evidence of the validity of the 
full-length SLEP, as measures of relatively closely 
related but distinguishable aspects of proficiency in 
English as a second language . 15 " ' 

General evidence regarding the structure of human abilities 
indicates that a positive correlational manifold can be expected to 
obtain among measures of ability generally. This applies to 
measures of second-language macroskills (e.g., Hale, 1986; Oiler, 
1983; Carroll, 1967): measures of different aspects of English 
proficiency can be expected not only to be positively correlated, 
but also to exhibit "moderate" to "moderately high" levels of 
intercorrelation (e.g., .50 to .70) in samples from general ESL 
populat ions, such as that being served in the LACCD. 

Grades earned by ESL students in courses of ESL instruction 
are widely accepted as validly indexing individual differences in 
"demonstrated competence" in the performance of varied tasks in 
proficiency-related domains that constitute foci of instruction — 
for example, reading and vocabulary, conversation, grammar, 
writing, or a combination of the foregoing. Accordingly, based on 
evidence and lines of reasoning developed above, scores on the 
shortened SLEP and the writing test can be expected to correlate 
positively with course grades. 

However, when test/grade correlations are involved, only 
accumulated empirical evidence can resolve questions as to the 
levels at which specified tests (or types of tests) typically 
correlate — hence, can be expected to be correlated — with course 



By inference from the observed levels of correlation, 
writing samples are being rated on the respective campuses with a 
"workable" degree of rank-order reliability. Internal consistency 
estimates available for test-development samples for SLEP Form 1, 
reported in Section I, are reported below for current reference: 
for item-type sections included in shortened SLEP, estimated 
reliabilities were .81 for One Picture, .89 for Dictation, .89 for 
Cloze, and .71 for Four Picture items. 
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grades in various use contexts. Moreover, as indicated in the 
preceding section, observed correlations among test variables in 
restricted samples are limited as bases for inferences regarding 
the correlations among the variables in general samples. 

• As shown in the preceding section, illustratively, 
observed correlations between scores on the writing 
samples and scores on the shortened SLEP in English 85, 
a placed sample, bear no resemblance to those observed in 
the general population involved. Similar distorting 
effects can be expected, within placed samples, for 
observed correlations of the placement tests with other 
measures, including course grades. 

This section reports on analyses designed to obtain and 
evaluate empirical findings regarding 

• levels and patterns of observed test/grade correlations in 
a representative array of ESL courses; 

• l£he extent to which both levels and patterns of coefficients 
for the placement variables are affected by nonvalidity-related 
statistical artifacts (e.g., sample standard deviations for the 
placement tests, sample size, differences in relative weighting of 
SLEP and Essay in placement composites) ; and 

• degree of consistency in observed levels of test/grade 
correlations across subgroups (e.g., gender, age, educational 
level, language) ; 

The findings indicate that scores on the shortened SLEP and 
the writing test, respectively, and as combined for placement 
purposes in different colleges, tend to be positively correlated 
with course grades, and that this is true for data aggregated by 
college, gender, age, language, and so on; also that both level and 
pat-terning of correlations for placement tests with grade in ESL 
courses are affected in predictable ways by differential 
restriction of range on the placement tests. 

Findings are presented and discussed following a brief 
overview of methodology and procedure. 



General Methodology and Procedure 

Analyses of test/grade correlations were conducted in a 
reduced sample made up students with "complete-data" records — that 
is, records with nonzero scores on the shortened SLEP, an essay 
rating, a score on the placement composite, and either a grade on 
the A-F scales or a pass/fail grade. Approximately 60 percent of 
the originally assessed population was included in the reduced 
sample . 
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Some 160 "unit" samples were represented in the 
complete-data sample — that is, samples representing 
courses in which students were actually graded (e.g., 
students earning a grade in English 85, City College, 
Spring 1991, constituted a unit sample, those earning 
grades in English 85 in Fall 1991 constituted a second 
unit sample, and so on. 

In several of the unit samples — all were courses graded on the 
pass/fail system — there was no variability in the grade distribu- 
tion; either all students passed or all students failed. These 
unit samples were excluded from the analysis, as were unit samples 
with fewer than 10 cases. The foregoing process resulted in the 
identification of 140 unit samples, including some 5,635 students. 
When students were classified within colleges by course, ignoring 
distinctions between terms of enrollment, the result was a total of 
59 course-level samples — data for students in the respective 
courses, aggregated across terms of enrollment. 

To permit assessment of test-criterion relationships in the 
the 59 course-level samples described generally above, while 
avoiding interpretive complications associated with possible 
differences in grading standards within and across colleges, all 
variables were "standardized" using unit-sample parameters 
(summarized in Appendix C, for 140 samples involved) . 

More specifically, distributions of scores on all 
variables (including grades on the five-point A-F scale 
[4,3,2,1,0] or the pass/fail [1,0] scale) were z-scaled, 
using unit-sample parameters — that is. scores on each 
variable were expressed as deviations from the unit- 
sample mean on the variable, and then divided by the 
corresponding standard deviation. Variables were thus 
converted to a common scale, with zero mean and unit 
standard deviation (0.0, 1.0). 

From a methodological perspective, it is useful to note that 
when variables have been z-scaled as indicated above, coefficients 
computed using aggregated data for two or more intact unit samples 
are comparable to size-adjusted averages of corresponding unit- 
level coefficients. 

Unless otherwise noted, the correlation coefficients reported 
below reflect relationships between the criterion variable (z- 
scaled grade) , and z-scaled scores on the placement variables 
(shortened LC, shortened RC, shortened SLEP, Essay, and/or Comp 
[the composite of percent-right scores for LC, RC, and Essay, that 
was used to guide placement decisions]). 
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Analytical Procedures 



Test/grade coefficients were first computed for each of the 59 
course-level samples. Because scores on the shortened SLEP LC and 
RC sections were not used separately in placement, attention was 
focussed primarily on coefficients for the total score on the 
shortened SLEP (LC+RC) . 

1. To provide perspective regarding the level and range of 
test/grade correlations, and the extent to which level of 
association tended to be infuenced by range restriction and related 
factors, distributions of coefficients were developed for course- 
level samples classified according to corresponding test standard 
deviations. 

2. To assess expected differential restriction of range ef- 
fects, an analysis was made of distributions of coefficients for 
the shortened SLEP and the Essay, respectively, for samples from 
East and Pierce, and from City, respectively. 

Coefficients for SLEP were expected to be higher than 
those for Essay at East and Pierce (where SLEP was less 
heavily weighted than the Essay in placement) , whereas 
the opposite was expected to be true for sampler from 
City (and other colleges where Esssay was less heavily 
weighted than was SLEP) . 

3. A third line of inquiry bearing on the influence of sta- 
tistical artifacts on observed levels of correlation involved re- 
gressing the observed coefficients on the corresponding sample 
standard deviations and sample size. 

All of the foregoing analyses involved observed coefficients 
and corresponding sample statistics for the 59 course-level 
samples. 

4. To assess general consistency of findings regarding test/ 
grade correlations across diverse samples, correlational analyses 
involving z-scaled scores on all variables were conducted for 
subgroups classified by college, gender, age, educational level, 
language, and so on. 

Finally, general estimates of levels of correlation for 
unrestricted samples are provided, based on the regression of 
course-level SLEP/grade, Essay/grade, and Composite/grade 
correlations on the corresponding standard deviations, sample size, 
and type of grading system (A/F vs. P/F: nominally coded "1" for 
AF system and "0" for P/F system) . These estimates are intended 
primarily to give added emphasis to the fact that restriction-of- 
range effects distort observed correlations involving the test 
variables, and limit generalizations about the validity or relative 
validity of the tests involved as applied in general populations. 
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Findings 



Results of the several lines of inquiry involving course-level 
coefficients are presented first; trends in test/criterion correla- 
tion in more general samples are then examined. 

Analyses Involving Course-Level Coefficients 

Table 4.1 reports observed course-level test/grade cor- 
relations for scores on the shortened SLEP (LC, RC, and SLEP 
[LC+RC], the writing sample (Essay), and composites .(Comp) that 
were used in placement, respectively; also corresponding course- 
level standard deviations, and data on sample size for the 59 
samples involved. Table 4.1 summarizes data only for the variables 
directly involved in the analyses reported below, namely, coeffici- 
ents and sample statistics for SLEP (LC+RC) , Essay, and Comp, 
respectively (see Appendix C for data on all variables) . 

Table 4.2 shows stem-leaf plots of test/grade coefficients for 
SLEP (LC+RC) , Essay, and Composite (composites used in place- 
ment) , for samples classified by relative degree of restriction on 
SLEP, as indicated by SLEP standard deviation (SD) . Three SD-ranges 
were defined arbitrarily to include about one-third of the samples 
in each range. Note that SLEP standard deviation in the total 
sample is approximately 19. 

Level of correlation decreases as SLEP standard de- 
viation decreases — true for the Essay and the composite, 
as well as for the SLEP itself. 

In evaluating the median coefficients for SLEP and Essay, 
respectively, keep in mind that placement was dependent more heav- 
xly on SLEP performance than on Essay performance in the great 
majority of samples under consideration — Essay was more heavily 
weighted in placement than was SLEP in decisions affected placement 
only at East and Pierce (11 of the 59 samples involved) . 

Differential restriction of range effects are clearly di- 
scernible in the results of analyses summarized in Table 4.3. 
Stem-leaf plots are shown for 22 samples from City (where SLEP 
received greater weight in placement) and 11 samples from East and 
Pierce (where the opposite obtained) . 

As expected, Essay/Grade coefficients tended to be 
higher at City than at East/Pierce, and SLEP/Grade 
coefficients tended to be higher at East/Pierce than at 
City. 

Related analyses. The foregoing analyses involving simple 
cross-tabulations of the basic data under consideration indicate 
that 



LACCD/SLEP Study 43 



S3 



1 

0) 
(0 

U & 
3 C 
O -H 
U TJ 
C 

c o 
•h a 
n 
a> « 
-o u 

n o 
o u 

o) a 

05 X! 
U +J 
3 

O C 

U *H 

o to 

■P 0) 
rH 

W £1 

0) (0 
rH -H 

XI u 
(0 10 
-H > 

u 

to a 

> X! 

+J 

■p 

a> o 
m 

a in 

o c 

(0 o 

rH -H 

a. 4-> 
<c 

0) > -P 

p a> o 

ffl Q 03 
C f-i 

-H U .3 
U) (0 w 

a) ts 

Q CO 

a c 

<M P RJ 

o ca 

Vi rH 

c - o 
o a u 
-h a> 

rH E 

0) « - 
OS W W 
0) 

rH rH 

. a> a 

f H t/1 

0) 

rH 

XI 

ro 
E-< 



CO 



rs. <j cm cm r> »- «— <3t>f- w om ^OO^-j-nt co 00 O 

Q.T0 

n o O *o»A^co*-«)*-m o m ^ m n C K in o »y ia io o K 

>. co o »- w co to o»- o aruo o f~ © © r^^^^oo <f <ono<o 

<D "D .... .... 

en «a *.*» *-* ** *— *- 

CO 
LU 

UO. *?mCM<0 NOr1CK)NtMN CO WsT <0 CO ID ^) N <0 <0 O CO 

(/»</) t» w- *- <<F" «r- •-■*-«- — *- «- 

$U tDC O O •-OO'-lAfNJ'O^ N-NmO> IT, O K) N O O N O tn O 

CM CM K» *~OCAJ*-«—'»-«J*- *~ O V— KJWMON^ %T O CM 

C_> .... .... 

>- 

« u. *r «© *- ooojnvdo'OO 0*000 k» © »- cm in ki -<o 

w ui cm cm «- <r-<v»-o»-^-0"— raoom *n cm »- c\j m «— co 

w .... .... 

LU tiii 
*~» CL 

Lu l- ooosr om'Cnooroo ocmio^o k> © *o © *- m %r »- cm n 

O—l OCMKICNi 00*-»-*-OK>'- O NNCMMOM M K» »- CM 

.C W 

(/) 1 1 ii 

C m cm cm ip»-\r»-'0>ywMn t> ^- »- cm **> cm «* •>« ki m, m »- co 

•-K'ON t> N. <0 (V fU M >0 « OlAM»1 muiOO^fO l>0 KlKIN 

CM *— cm «— 

DC CM K> U"* W<tfin«OK^r-pl ?0 Ut «0 -O M** Ifl <0 W vf in MJN Kl 

g COK>OttD«f-<- KcOOtOO*- CO CD CO CO «— 

u 

-5 0000 c>9>g>c><>*-*-*- 0000 m <> g> <> <> »- 9* <> «> c> »- 

m 0000 <>c>c>c>r>ooo 0000 o 6^ t> c> c> o o- <> c> &. o 

wwnnwoSo *x -*r r\j pom mm o mnmno 

VI 

—j "D vo «o w^^^K-f^f^i*- ** «} m <m ** cm 01 cm cm cm 1- ia mm in 

__i • — CO » « 

o • t* « — 

cj *- > 3 iu a. 



I 3 



• 13 
m v> 



1- a. _ 

Z -J u 



^Kt->-0>N^OCV'-ONOO«OB>OWt-0 CM Q> I 



n^8S<4 , inoKi4«-iniA^<rNonintMs)r- o. -r «— f*. o »- ■•o 



•TWM-<-MKiNmm»-fO'-Nm*---iMamfyw r\i m o »- o o o o o «- 



i-nmi»<aniniM<rinini'inih><-<r<ouMMin 4 ««o •»o<Oj;«in 

»->>0<-IAIVIXNNnniinNKNI>a'«l)N« •» O. w o> <fl IM t> i> o (> 



<: 

<: 
o 

CJ> 

H— 

CO 
LU 
CO 



OJrfnOO««0< 



t- r\j n in 'O 



ki «- «J fo •* in <o K. «5 p> o «r 0 «f m >fi N. <v »o m <iiA< 

nnninnnmnnnn»»»»»»o<>--'- m o. o> » 000000 
^•O'O^^^'O'O'O'O'O&'W'^^O'd^&'OPP *o i> ^ o- 000000 



a -* 



LACCD/SIiEP Study 



0 

ERIC 



44 



CD 



Table 4.2. Distributions of Course-Level Correlation Coefficients 
for Samples Classified According to Degree of 
Restriction on Shortened SLEP 



SLEP standard deviation 

11.5 + 9.3-11.4 Less than 9.3 

SLEP/Grade coefficients 

Stem Leaf Stem Leaf Stem Leaf 

.5 5 .5 — .5 

.4 15 .4 26 .4 

.3 123459 .3 045 .3 05 

.2 034667 .2 00268 .2 346 

.1 0036 .1 2346 .1 0123 

• 0 — .0 1359 . 0 014679 

-.0 — -.0 28 -.0 2356 

-.1 — -.1 3 -.1 3 
-.2 

-.2 — -.2 

Mdn .27 .15 .08 

Essay/Grade coefficients 



.5 


2 


.5 




.5 




.4 


118 


.4 




.4 




.3 


5568899 


.3 


023 


.3 


0229 


.2 


0136 


.2 


1156 


.2 


0467 


.1 


5779 


. 1 


0235677 


.1 


023679 


. 0 




.0 


2 


.0 


0 


-.0 




-.0 


88 


-.0 


268 


-.1 




-.1 




-.1 


69 


-.2 




-.2 


2 


-.2 




Mdn 


.35 




.17 




.165 


ite/Grade coefficients 








.5 


2 


.5 


2 


.5 




.4 


118 


.4 


002 


.4 


4 


.3 


5568899 


.3 


3 


.3 


9 


.2 


0136 


.2 


0113589 


.2 


4455 


.1 


5579 


.1 


124788 


.1 


12589 


.0 




. 0 


0 


.0 


799 


-. 0 




-.0 




-.0 


226 


-.1 




-,1 




-.1 


5 


-.2 




-.2 


0 


-.2 




Mdn 


.35 




.21 




.125 


N 


19 




20 




20 



Note . To read coefficients, combine "stem" and "leaf" values. 
For example, under Composite/Grade, .52, .41, .41, .48 and so 
on. 
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Table 4.3. Plots of Course-Level Test/Grade Correlation 
Coefficients for Samples from City, and East and Pierce, 
Respectively 



CITY 


SIiEP 




l_i O O U J 






Stem 


Leaf 


Stem 


Leaf 


Stem 


Leaf 


.5 


5 


.5 


— 


.5 


2 


.4 


1 


.4 


34 


.4 


0148 


.3 


1255 


.3 


0122334579 


.3 


588 


.2 


3668 


.2 


5567 


.2 


01344455 


.1 


2334 


. 1 


0237 


.1 


24578 


.0 


14579 


. 0 


0 


.0 


9 


-.0 


225 


-.0 




-.0 




-.1 




-.1 


6 


-.1 




-.2 




-.2 




- . 2 




No. samples =22 










Mdn r 


.135 




.305 




.240 


EAST. 


PIERCE 










Stem 


Leaf 


Stem 


Leaf 


Stem 


Leaf 


.5 




.5 


— 


.5 


— 


.4 


2 


.4 


0 


.4 


012 


.3 


0459 


. 3 


34 


. 3 


359 


.2 


0367 


.2 


15 


.2 


5 


.1 


1 


.1 


3467 


.1 


9 


.0 


1 


.0 




.0 


6 


-.0 




-.0 




-.0 


2 


-.1 




-.1 


9 


-. 1 




-.2 




-.2 


2 


-.2 


0 


No. samples = 11 










Mdn r 


.270 




.191 




.290 



Note. To read coefficients, combine entry in "stem" column with 
successive entries in "leaf" columns. For example, SLEP 
(.55, .41, .31, .32 and so on). 
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(a) level of test/grade correlation varies markedly across 
samples, 

(b) level of correlation tends to decrease with degree of 
restriction on placement variables (restriction on SLEP used 
illustratively) , and 

(c) differential restriction of range effects are present, 
thus complicating validity-related interpretation of the observed 
coefficients for the variables involved (Essay and SLEP daca used 
illustratively) . 

Additional evidence regarding the extent to which level of 
test/grade correlation across the 59 samples covaried with non- 
validity-related sample statistics (standard deviations and size) 
is provided by results of analyses involving the simple correlation 
of sample-level coefficients for SLEP, Essay, and the placement 
composite (Comp) , respectively, with the corresponding sample 
standard deviations and sample size. 

Table 4.4 shows correlations reflecting the extent to which 
differences in level of test/grade relationship (sample test/grade 
coefficients) were associated with corresponding differences in 
sample dispersion on the test involved. Data were analyzed sep- 
arately for ESSAY-restricted samples (East/Pierce) SLEP-restricted 
samples (all other colleges) , the classification based on relative 
weighting of the two measures in placement composites, used in the 
distributional analyses reported above. 

• In the overall analysis, regardless of the particular 
validity coefficient involved, SLEP standard deviation 
was the best single predictor of level of relationship. 

• However, for SLEP-restricted samples, the best predict- 
or of level of SLEP coefficient was Essay standard dev- 
iation, and for ESSAY-restricted samples, SLEP standard 
deviation was the best predictor of level of Essay 
coefficient. 

• As expected from results of the tabular analyses, 
above, Essay correlated more closely with grades when 
SLEP was emphasized in placement, and the opposite was 
true when Essay was emphasized. 

Test/Grade Coefficients in Diverse Subgroups 

The foregoing analyses focus attention on distributions of 
coefficients, calling attention to variability in observed test/ 
grade correlations as well as central tendencies. The findings also 
point up nonvalidity-related variables that influence observed 
outcomes, complicating validity-related interpretive inferences. 
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Table 4.4. Correlation of Observed Coefficients for Designated 
Placement Variables with Corresponding Sample Standard 
Deviations and Sample Size: Analyses Across 59 Samples, 
Weighted by Sample Size 

Sample statistic 

Lze Mean 
coef f . 
( r ) 

(.17) 
(.18) 
(.22) 

(N=5635) 

(.14) 
(•18) 
(.20) 

(N=4680) 

(•28) 
(.17) 
(.26) 

(N= 950) 

* Correlation of variable with grades in the respective 
samples — the criterion variable. 

** SLEP-i ^stricted samples are from City, Harbor, Mission, Trade 
Valley and West; ESSAY-restricted samples are from East and Pierce 
Data for samples were weighted by sample size. 



Test/grade 


SLEP 


Essay 


Comp 


coefficient 


Sd 


Sd 


Sd 


N 


in sample 


r 


r 


r 


r 


(All Samples) 










siiEP coefr.* 


. 53 


.18 


.45 


-.22 


ESSAY coeff. 


.19 


. 14 


.26 


-.03 


COMPOSITE coeff. 


.46 


.24 


. 34 


-.16 


Mean (statistic) 


10.6 


0.92 


8.3 


135 


fSLEP-restricted) ** 










SLEP 


.31 


.46 


.62 


-.20 


ESSAY 


.22 


.37 


.29 


-.09 


COMPOSITE 


.36 


.47 


.55 


-.27 


Mean (statistic) 


10.5 


0.95 


8.3 


140 


f ESSAY-restricted) ** 








SLEP 


.13 


-.21 


-.11 


.03 


ESSAY 


.63 


-.07 


.17 


.24 


COMPOSITE 


.50 


-.31 


-.06 


.31 


Mean (statistic) 


11.4 


0.76 


8.2 


112 
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Given these circumstances, emphasis in further analysis of the 
data was primarily descriptive, concerned primarily with providing 
evidence bearing on the general consistency of correlational 
outcomes across colleges and subgroups defined by gender, language, 
educational level, and age, respectively. For general perspective, 
findings are presented separately by grading system (A-F versus 
Pass/Fail) . 

Table 4.5 shows college-level findings: average correlations 
with course grade for all the placement variables — that is, 
shortened SLEP listening (LC) , reading (RC) , and combined (SLEP = 
LC+RC) scores, Essay score, and score on the placement composite 
(Comp) . Colleges are listed in descending order with respect to 
average course-level standard deviation on the shortened SLEP. 

• Average coefficients for all placement variables tend 
to be noticeably lower at Valley and West, with greatest 
within-course restriction on SLEP (and incidentally on 
other variables), than at the other colleges. 

• The overall Essay/Grade coefficient was lower when P/F 
grades were involved (r = .07) , than when A/F grades were 
employed (r = .20). This outcome was influenced strongly 
by the very low Essay/Grade correlation (.01) in the 
large sample from Mission. 

• No interpretable "relative validity" pattern is dis- 
cernible: in three analyses the highest coefficient was 
that for a shortened-SLEP section (RC) , in two instances 
this was true for the combined SLEP score, in three 
instances for Essay, and in the remaining four instances, 
for the Composite. 

Findings for the designated subgroups, presented in Table 4.6, 
indicate that coefficients for the respective subgroups are 
generally positive; also that differences in level are roughly 
comparable to those observed across colleges; and so on. 



An Exercise in Estimation 

Findings reviewed thus far indicate that differences across 
courses in observed test/grade correlations are associated with 
differences in nonvalidity-related characteristics of the samples 
in which the coefficients are computed, including sample standard 
deviations on test variables and sample size. The extent to which 
each of these factors, considered separately, is associated with 
particular test/grade coefficients was indicated in Table 4.4, 
above. To assess the extent to which these variables, and type of 
grading system, tend to account for differences in observed course- 
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Table 4.5. Pooled Within-Course Correlations of Tests with Grades, 
by College and Type of Grading System (A-F vs. Pass/Fail) 



College 


N 


LC 


RC 


SLEP 


ESSAY 


COMP 


Mission (PF) 


732 


.21 


.19 


.23 


.01 


.21 


East (AF) 


619 


.15 


.27 


.26 


. 15 


.23 


Pierce (AF) 


331 


. 22 


. 32 


. 31 






Trade (AF) 


83 


.21 


.23 


.28 


.20 


.32 


(PF) 


87 


.24 


.25 


.31 


.23 


.34 


Harbor (AF) 


213 


.12 


.28 


.26 


.17 


.26 


City (AF) 2 


,011 


.13 


. 14 


. 25 


. 27 


.25 


(PF) 


326 


.01 


.11 


.09 


.13 


.15 


Valley (AF) 


877 


.11 


.07 


.10 


.11 


.14 


(PF) 


95 


-.05 


.13 


.06 


.11 


.08 


West* (AF) 


260 


. 06 


-.02 


.02 


.08 


.06 


Total (AF) 4 


,394 


.13 


.15 


.17 


.20 


.22 


(PF) 1 


,240 


.13 


.16 


.18 


.07 


.19 



Note. Colleges are listed in descending order with respect to 
average course-level SLEP standard deviation. 

* Includes only those students with both LC and RC scores on 
the shortened SLEP, primarily from the Spring 1991 term. 
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Table 4.6. Correlations of Tests with Grades in Designated 
Samples: By Grading System (AF=A to F vs. PF = 
Pass/Fail) 



Group 


N 


LC 


RC 


SLEP 


Essay 


Comp 


Gender 














Males (AF) 


1, 712 


. 08 


. 16 


1 4 


. 16 


. JL O 


Females 


2 , 682 


. 18 


. 17 


.20 


.21 


.26 


Males (PF) 


555 


.20 


.30 


.31 


.04 


.31 


Females 


685 


.08 


.06 


.08 


. 08 


.10 


Lanauacre 














Soanish ( AF) 


1, 471 


. 16 


17 




1 7 


9 9 


Armenian 


782 


. 14 


. 15 


1 7 

• JL / 


• aO 


9 6 


Korean 


379 


.05 


.16 


.13 


.23 


.21 


Chinese 


359 


.05 


.19 


.17 


.16 


.18 


Japanese 


264 


.12 


.29 


.26 


.03 


.19 


All Other 


1, 139 


.12 


.17 


.18 


.21 


.24 


Spanish (PF) 


555 


9 0 


7 n 
■ *j \j 


« jl 


n a 


• Jl 


Armenian 


71 




9 4 


1 7 
. -L -J 


1 0 
• xz 


• jlD 


Korean 


86 


-. 03 


• JL JL 


no 




• JL Z7 


Chinese 


13 


-.14 


. 12 


m 

* W J- 


9 4 


• JL JL 


Japanese 


24 


.41 


.39 


.48 


-.11 


.36 


All Other 


273 


.21 


.21 


.25 


.10 


.27 


Educational Level 












NonHSgrad(AF) 


870 


. 14 


. 13 


. 15 




1 7 


HS grad 


3, 147 


. 12 


. 15 


. 16 


.20 




BA or higher 


377 


.23 


.20 


.25 


. 17 


98 


NonHSgrad(PF) 


517 


. 12 


. 12 


. 14 


. 10 


1 7 


HS grad 


609 


. 13 


.17 


. 19 


. 04 


. 19 


BA or higher 


114 


.16 


.21 


.23 


.05 


.23 


Age 














< 22.5 yrs(AF) 


1, 101 


.09 


.19 


.17 


.16 


.20 


22.5-30.5 yrs 


1,560 


. 10 


.12 


.13 


.18 


.18 


3 0.5 yrs plus 


1,733 


.19 


.16 


.21 


.23 


.27 


< 22.5 yrs(PF) 


232 


.12 


.20 


.19 


.11 


.22 


22.5-30.5 yrs 


494 


.15 


.14 


.18 


.02 


.16 


30.5 yrs plus 


514 


.12 


.17 


.18 


.10 


.21 
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level correlations, three stepwise multiple regression analyses 
were conducted, each involving the regression of a particular test/ 
grade coefficient (treated as the dependent variable) on the fol- 
lowing nonvalidity-related variables: standard deviations for 
SLEP, Essay and the Composite, respectively, sample size, and type 
of grading system (nominally coded 1 = AF system and 0 = PF 
system) . 

Salient findings of these stepwise analyses are shown in Table 
4.7. In evaluating the findings generally, it is useful to note 
that negative coefficients for the composite reflect suppression 
effects: the composite standard deviation was positively related 
to course-level coefficients (see Table 4.4, above). The negative 
weighting shown, reflects the close relationship between restric- 
tion on the SLEP and restriction on the Composite, and the fact 
that SLEP standard deviation was more closely related to the cri- 
terion (course-level correlation) than was the composite. However, 
the negative weights for sample size are directly interpretable as 
indicating higher coefficients tend to be found in smaller samples. 

With the foregoing in mind, it can be seen in the table that: 

standard deviation on the shortened SLEP was the most 
highly weighted variable not only in analyses involving 
course-level SLEP/grade coefficients but also in anal- 
yses involving Composite/grade coefficients; 

the most important predictor of level of Essay/grade 
correlation was type of grading system; SLEP standard 
deviation provided the next largest amount of inform- 
ation; and 

the Composite standard deviation did not contribute to 
level of Essay/grade correlation in the stepwise regres- 
sions, after other variables were entered. 

The corresponding multiple correlation coefficients are also 
shown in the table. Results of the regression analyses provide a 
direct empirical basis for estimating levels of correlation in 
otherwise comparable, but unrestricted samples — that is, regression 
equations for estimating course-level coefficients for the respec- 
tive tests from corresponding sample statistics. 
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Table 4.7. Results of a Stepwise Regression of Course- 
Level Coefficients on Designated Non- Validity-Related 
Sample characteristics: Data Weighted by Sample Size 

Dependent Standard partial regression weights* 

coefficient Standard deviation Grading size 

( R ) SLEP Essay Comp A-F n 

SLEP(r) (.63) 1.00 .17 -.48 .14 -.27 

Essay(r) (.47) .57 .17 ... 1.00 -.25 

Composite (r) (.59) 1.00 .08 -.76 .20 -.18 



♦These are conventionalized weights — that is, coefficients in 
each row are shown as proportions of the largest coefficient. 

— Indicates variable did not enter the stepwise regression. 
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The raw score regression weights and contant terms for the 
three equations were as follows: 

Raw-score regression weights for 
variable 



Estimated 


SLEP 


Essay 


Comp 


AFsys 


Size 


Constant 


coefficient 


sd 


sd 


sd 






SLEP (r. est) 


.051 


.097 


-.025 


.043 


-.0005 


-.2187 


ESSAY (r. est) 


.013 


.064 




.136 


-.0002 


-.1031 


COMP(r.est) 


.063 


.214 


-.048 


.079 


-.0004 


-.2582 



We know that standard deviations for SLEP, Essay, and 
Composite are, respectively, 19.3, 1.5, and 16.8, in the general 
population. To estimate coefficients for the shortened SLEP, the 
Essay, and the Composite, respectively, for course-level samples 
with the foregoing standard deviations, the three standard devi- 
ations were inserted in the respective raw-score regression equa- 
tions, along with the average N (135), and the nominal code of "l" 
for course grading under that system. P/F-course estimates were 
similarly generated. 

Results are shown below: 



Estimated coefficient 
Grading Short Essay Composite 

system SLEP 

A/F system .47 .35 .50 

P/F system .43 .21 .42 



These estimates reflect the observed empirical relationships 
between course-level correlation coefficients and sample standard 
deviations for the respective placement variables. 

Generally speaking, these findings point up the extent to 
which the observed average course-level coefficients that have been 
reviewed in this section, tend to reflect effects associated with 
restriction of range on the tests, sample size, and type of grading 
system. They give added emphasis to the axiomatic proposition that 
observed levels of test/grade correlation need to be evaluated in 
light of the possibility that 

"... the magnitude of the correlation (between tests 
and grades) may vary as a function of the degree to 
which a test was used to place students in the course 
under investigation and/or the variation in grading 
standards across classrooms" (California Community 
Colleges, 1992: p. 20). 
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SECTION V. RECAPITULATION AND EVALUATION OF FINDINGS 

As indicated in Section I, the Secondary Level English 
Proficiency (SLEP) test was initially selected for use in placement 
testing in the LACCD by ESL professionals (the LACCD ESL Committee) 
through a process that included 

(a) a comprehensive, comparative assessment of the relative 
appropriateness (for LACCD purposes) of nine commercially available 
tests of ESL proficiency, and 

(b) consideration of the findings of a specially commissioned 
study (Butler, 1989) of the performance of samples of LACCD 
students on all SLEP items — findings indicating that the level of 
difficulty of the test was generally appropriate for the students 
involved, and that average scores tended to covary positively with 
ESL placement level. 

To meet purely pragmatic interest in reducing the amount of 
testing time needed to administer the SLEP (approximately 85 
minutes) and a locally developed writing test (30 minutes), a 
decision was made to administer only a portion of the full-length 
SLEP, namely, two of four item-type susections involving listening 
comprehension items, and two of four item-type subsections 
involving reading comprehension items. 

The present study was undertaken with the encouragement and 
support of the SLEP Testing Program at Educational Testing Service, 
and the collaboration of the LACCD District Office, represented by 
Ms. Rebecca Tillberg, 

(a) to examine patterns of performance on the shortened SLEP 
and the writing tests in the general ESL population and in selected 
demographic subpopulations; 

(b) to investigate levels and patterns of concurrent rela- 
tionships among scores on the components of the LACCD placement 
battery ; 

(c) to obtain base-line empirical evidence regarding observed 
levels of correlation between scores on the shortened SLEP test, 
the writing test, and the placement composite, on the one hand, and 
student performance in ESL courses, as indexed by grade earned (a 
grade on the "A-F" scale, or a Pass/Fail grade) , on the other, by 
course and by college, and in various subgroups (e.g., gender, 
educational level, age, language), and 

(d) to provide an analytic assessment of the extent to which 
observed relationships in placed samples are influenced by non- 
validity-related factors, especially restriction of range on the 
tests that were used to place students. 
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Findings pertaining to these objectives — and the more general 
objective of extending evidence of the validity of the full-length 
SLEP test in the LACCD context — are summarized briefly below. 



Summary 

The ESL population being assessed with the shortened SLEP is 
extremely heterogeneous wiLh respect to age, educational level, 
ethnicity, and language background. 

• The average student is about 29 years of age; based on 
data of record on date of birth, some students were less 
than 13 years old and others were over 65 years of age 
when assessed in the LACCD. 

• The majority of students hold secondary-school 
diplomas earned in schools located outside the United 
States, but the sample includes individuals who have not 
completed secondary school — some enrolled in adult 
education courses, some as special students — as well as 
individuals with bachelor's or higher degrees. 

• The largest linguistic subpopulation is made up of 
native-speakers of Spanish (about 44 percent of the 
total) ; none of the other directly identified language 
groups accounts for much more than 10 percent (11 
percent reported Armenian, and less than 1 percent 
reported Filipino) . 

• The shortened SLEP sections are at psychometrically 
efficient levels of difficulty for the general LACCD ESL 
population as well as for subgroups that differ markedly 
with respect to age, educational level, language 
background, and self-reported ethnic group membership. 

• Results of an analysis of trends in mean scores on the 
shortened SLEP and the Essay, respectively, across 
subgroups classified by educational level indicate that 
the two means covaried directly. Similar patterns of 
covariation are discernible for the two test means 
across other demographic subgroups. Generally speaking, 
demographic subgroups with higher (lower) means on the 
SLEP tend to have higher (lower) means on the writing 
test. 

• Analyses of concurrent relationships between shortened 
SLEP scores and scores or. the writing tests were 
moderate to strong in each college setting: within- 
college correlations between scores on the locally 
developed writing tests and scores on the shortened SLEP 
centered at about .60, varied between roughly .5 and .7 
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across colleges. These correlations were in general 
(pre-placement) samples. 

The relationship between the shortened SLEP reading 
comprehension score and Essay rating typically was 
somewhat stronger than that observed for the shortened 
SLEP listening comprehension score with Essay, con- 
sistent with expectation — that is, it is plausible that 
reading and writing skills should tend to be somewhat 
more closely related than are listening and writing 
skills. 

At the same time, coefficients for total score on the 
shortened SLEP test (LC + RC) were somewhat larger than 
those for RC alone, indicating that each of the two 
sections is providing some unique information regarding 
aspects of the ability being measured by the writing 
test — a face valid measure of ability to write compre- 
hensibly in English. 

Levels of conconcurrent relationship observed for 
college-level samples were also observed in the results 
of analyses conducted by college, illustratively, for 
Spanish-speaking and Korean-speaking subgroups, respec- 
tively. 

The foregoing findings provide strong empirical evidence of 
concurrent- and discriminant-validity properties for the shortened 
LEP and the locally developed writing tests. 

In this connection, it is considered particularly noteworthy 
that the levels of concurrent relationship observed in the LACCD 
context, between scores on the shortened version of the SLEP and 
scores on local writing tests (with attendant differences in topic, 
rating procedures, and so on) , are comparable to levels that have 
been found to obtain between scores on the Test of English as a 
Foreign Language and the Test of Written Expression (writing 
samples elicited under standard conditions, scored under controlled 
conditions by at least two raters, and so on) — true for, 

(a) samples tested in developmental research (Carlson, 
Bridgeman, Camp, and Waanders, 1985) involving the TOEFL and 
prototypical versions of the TWE, and 

(b) samples taking both the TOEFL and the TWE tests under 
fully rperational conditions (see ETS, 1992b) . 

The TOEFL/TWE samples, of course, are more highly selected 
than is the educationally heterogeneous LACCD sample. However, the 
LACCD findings attest to the power of direct observation of samples 
of pertinent behavior in ESL proficiency assessment. 
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Analyses of levels and patterns of correlation of the 
placement tests with course grades, involving 59 course-level 
samples, were complicated due to the need to assess the influence 
of nonvalid-ity-related factors on both levels and patterns of 
observed test/ grade coefficients. The generalizations outlined 
below reflect findings reviewed in detail and evaluated in Section 
4. 

Scores on the shortened SLEP and the writing test, 
respectively, and as combined for placement purposes in 
different colleges, tend to be positively correlated 
with course grade, but there is substantial variability 
across courses in level of relationship. Distributions 
of observed course-level coefficients for the respective 
tests tend to center at about the .2 level, with 
substantial variability (ranging upward from negative 
values to values above .5). 

Essay scores received relatively more weight than did 
SLEP scores in placement at East and Pierce, and in 
samples from those colleges, average coefficients were 
higher for the SLEP than for the Essay. SLEP was 
weighted relatively more heavily than Essay in other 
college samples, and in these samples, Essay coeffici- 
ents typically were higher than SLEP coefficients. Such 
findings are interpretable as reflecting differential 
restriction of range effects. 

Differences across 59 course-level samples, with 
respect to level of test/grade coefficients for the 
placement tests (that is, SLEP/grade coefficients, 
Essay/grade coefficients, and Composite/grade coeffici- 
ents, respectively) , were found be associated relatively 
strongly with nonvalidity-related sample character- 
istics: differences in the corresponding sample stan- 
dard deviations, type of grading system employed (A/F 
versus Pass/Fail) , and sample size. 

The respective test/grade coefficients for the 59 course-level 
samples, treated as dependent variables, were regressed on the five 
nonvalidity-related variables. Results indicated that 

sample standard deviation for the shortened SLEP was 
the most highly weighted variable not only in analyses 
involving course-level SLEP/grade coefficients but also 
in analyses involving Composite/grade coefficients; 

the most important predictor of level of Essay/grade 
correlation was type of grading system; SLEP standard 
deviation provided the next largest amount of informa- 
tion ; and 
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• the Composite standard deviation did not contribute to 
level of Essay/grade correlation in the stepwise regres- 
sions, after other variables were entered. 

Regression equations developed in the foi-egoing analysis 
provided a basis for estimating test/grade coefficients (SLEP/ 
grade, Essay/Grade, and Composite/grade coefficients, respective- 
ly) , assuming no selection on the variables involved, values for 
general sample standard deviations were substituted for the re- 
stricted values in the respective equations. Estimated values and 
observed values are shown below. 



Test/grade correlations 







Short SLEP . 


Essay 


Composite 


A/F 


sys 


(estimated) 


.47 


.35 


.50 


A/F 


sys 


(observed) 


.17 


.20 


.22 


P/F 


sys 


(estimated) 


.43 


.21 


.42 


P/F 


sys 


(observed) 


.18 


.07 


.19 



Generally speaking, the findings involving test/grade rela- 
tionships that have been reviewed should be thought of primarily as 
a basis for empirical assessment of the extent to which observed 
course-level test/grade coefficients generally, are influenced by 
nonvalidity-related factors, and are thereby limited as bases for 
inferences regarding "levels or patterns of validity" for the tests 
involved when applied in the general population under consider- 
ation. 



In more general analyses, positive test/grade correlations 
varying around the observed values noted above were found to obtain 
m samples aggregated by gender, age, educational level, and 
language, respectively. 

Regarding the Writing Tests 

• Pooled within-school SLEP/Essay coefficients typically 
were larger than the "total sample" coefficients, the 
latter being attenuated by inconsistent ordering of 
school means on the respective tests — an anomalous 
finding, in view of the systematic, positive within- 
school relationships observed. 



College differences in "rating standards" are 
suggested by the results of analyses of differences 
across colleges with respect to mean discrepancies 
between observed Essay rating, and Essay rating 
estimated from SLEP performance, using a total-sample 
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regression equation. Mean discrepancies (residuals) 
were substantially more prononced in analyses by 
college, than in analyses for subgroups by gender, 
language, educational level, and age, respectively, 
aggregated across colleges. 

• In view of apparent differences across sites in 
"rating standards," pending resolution of questions 
regarding comparability of Essay scores across colleges, 
future investigations of subgroup differences in the 
relative development of Essay-assessed "writing ability" 
and SLEP-assessed skills need to be conducted at the 
college level. 

These and related issues constitute meaningful topics for 
further inquiry. The findings suggest the need for systematic 
study of the rating process on each campus, including the collec- 
tion of data needed to evaluate degree of agreement among raters 
with respect to level as well as rank-order. 



Concluding Observations 

The findings of this collaborative undertaking provide direct 
empirical evidence that the shortened SLEP and locally developed 
writing tests are providing valid information regarding related 
aspects of ESL proficiency in the demographically diverse ESL 
student population being served by the LACCD; and the findings 
logically extend available evidence supportive of the validity of 
the Secondary Level English Proficiency Test for ESL assessment 
purposes in the LACCD and elsewhere. 

Considered in light of the rigorous screening and evaluation 
of available psychometric options that resulted in SLEP's initial 
selection by the ESL Committee, and other evidence of SLEP's 
validity cited herein including the generally positive findings 
reported by Butler (1989), regarding SLEP's appropriateness for ESL 
assessment purposes in the LACCD, the findings that have been 
reviewed herein support and extend the working proposition advanced 
at the outset, namely, that 

• the SLEP is a valid test of psychometrically distinct, 
albeit closely related, aspects of acquired proficiency 
in English as a second language: the ability to 
comprehend utterances in English, and the ability to 
read and comprehend the substance of material written in 
English, and 

• the SLEP can be expected to provide reliable and valid 
information regarding these abilities, when used for ESL 
assessment purposes by secondary schools, colleges, and 
other institutions, and 
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• given the demographic diversity of the LACCD samples 
in-volved in this study, the findings extend evidence 
that the SLEP can be used validly with local ESL student 
populations that are quite heterogeneous with respect to 
age, educational level, language background, and 
national origin. 

The local writing tests and the shortened SLEP appear to be 
providing an effective basis for placing students, within time 
constraints that appear to be considered necessary, from an 
administrative perspective. 

As noted by Butler (1989) , some cost in terms of diminished 
reliability and validity undoubtedly is entailed by the use of only 
a selected portion of the test. Available evidence bearing on this 
important question is limited, but consistent with the validity- 
cost assumption. 

In a cooperative study currently in progress (Wilson, 1993) , 
involving large samples of Japanese-speaking students being asses- 
sed for ESL placement at Temple University- Japan, LACCD-parallel 
subscores were computed post hoc from data available for the full- 
length SLEP. . Correlations between these scores and ratings of 
speaking ability and writing ability, respectively, were approxi- 
mately . 05 correlation points lower than correlations for cor- 
responding full-length scores, which centered around .6. 

• Rudmann (1991) , at Irvine Valley College, reports 
observed correlations centering around .40 between 
grades in ESL courses and scores on the full-length 
SLEP. 

Analyses involving the full-length SLEP test would be needed 
in order to address questions of comparative validity: comparative 
validity of the current shortened, LACCD version of the SLEP, and 
the full-length test, and/or comparative validity of the particular 
SLEP item types selected versus those not selected, and so on. 16 



In connection with the latter issue, it is noteworthy that 
one LACCD college (West) opted to use only the shortened SLEP 
reading section, with the writing test, for placement. More 
generally, in reporting on the basic SLEP validation study, 
Stansfield (1984) observed that users might elect to test either 
listening comprehension or reading comprehension in some 
circumstances (e.g., listening comprehension for level-placement 
decisions involving courses emphasizing the development of 
conversational skills) . An evaluation of experience at West, where 
only a score on items from the SLEP "reading comprehension" ability 
domain is used, would be useful. 
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• One empirical approach to such an assessment that 
offers a variety of pertinent data-generating pos- 
ibiiities would involve administering the full-length 
SLEP (Form 1 or one of the other two equated forms) as 
an exj-t test, to students who were tested for placement 
with the shortened version. The resulting item -level 
data for both tests would provide a basis- for assessing 
average (net) change in performance on the particular 
item- types now being used, and ^.ake it possible to 
evaluate the relative validity of all eight SLEP item 
types. Moreover, the end-of-course distributions of 
scores are needed to establish measured levels of pro- 
ficiency associated with particular ESL instructional 
sequences . 

Generally speaking, the use of more extensive and/ or compre- 
hensive testing procedures in ESL placement can reasonably be ex- 
pected to be accompanied by benefits attendant upon reduced inci- 
dence of perceived misplacement: for example, reduction in the 
educational and administrative costs associated with course chang- 
es, improved satisfaction with results of the placement process on 
the part of both teachers and students, and improved retention of 
students in the ESL program. 

A frrmal evaluation of the ESL placement process would provide 
empirical evidence that is pertinent to cost/benefit analysis, as 
well as to an overall evaluation of the ESL placement program in 
the LACCD r 
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APPENDICES 



Appendix A 

A.l. Holistic Scale for Rating Writing Samples in the LACCD 
(from Butler, 1989) 

A. 2. Data on Standardized English Proficiency Test Evaluated 
by the ESL Committee (from Butler, 1989, Appendix) 

A. 3. Illustrative SLEP Items 



Appendix B 

B.la. Distributions of Scores on Variables Used in Placement, 
for All LACCD Students Assessed: Spring 1991 through 
Spring 1992 

B.lb. Descriptive Statistics for Study Variables, by College 

B.2. Descriptive Statistics for Placement Variables, by College 
and ESL Level (Last Course Enrolled) : Classification of 
Courses (Other than ESL 400) According to City College Flow 
Chart (see text, Exhibit A) 

B.3. Mean Scores on Placement Variables, by Language Group and 
College: All Students Assessed 

B.4. Relationship Between Placement Variables and Selected 
Demographic Variables, By College 

B.5. Results of Regression of Essay Rating on SLEP LC and RC in 
the All-LACCD Sample: All Students Assessed (Basis for 
Residual Analyses, Section 3) 



Appendix C 

C.l. Descriptive Statistics for Placement Variables in Unit 
Samples Samples (Students Classified by Course and Term 
{ "SAMP" ] : Parameters Used for Z-Scaled Transformation of 
Test Variables) 

C.2. Descriptive Statistics for Placement Variables, in 
Samples Aggregated by Course, Across Terms of Enrollment: 
By College and Subject: 59 Course-Level Samples 

C.3. Correlations of Placement Variables with Course Grade, 
and Standard Deviations of the Variables in Corresponding 
Samples, Classified by College and Subject 
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Appendix A.l. Holistic Scale for Rating Writing Samples in the 
IACCD 



LOS ANGELES COMMUNITY COLLEGE DISTRICT 
E5L KATIKG SCALE 

0 Off the topic 

1 Organization fe Coherence — one or more sentences or fragments unable 

to supply details, examples, or elaboration in response to the prompt 
Mechanics— obvious lack of control of basic syntax, morphology, spelling. 

2 Organization & Coherence— a loose collection of sentences or fragments 

(short and elliptic or long and unclear) vaguely related to the prompt. 
May also appear as a collection or list of "notes" on the topic 
Mechanics— may lack control over basic syntax, morphology, spelling. May 
exhibit an oral" style— use of "gonna" (for going to), etc 

3 Organization & Coherence— generally stays on topic but sentences show 

little development. Essay may ramble or skim through a number of general 
examples without Making them or elaborating upon them. 
Mechanics— may lack control over grammatical conventions or exhibit " oral" style. 

4 Organization & Coherence— a large paragraph or several shorter paragraphs, 

showing more organization and development than a 3. One or more sentences 
may cluster around a main point without necessarily making it clearer, but 
usually, a clear cause and effect relationship is expressed. 
Mechanics— may contain many surface errors which are easily editable. 

5 Organization & Coherence— more developed than a 4; essay supplies 2-3 

sentences of specific detail to explain each example or explores a single 
example in depth; good use of transitions and Unking words. 
Mechanics— may contain many surface errors which are easily editable. 

6 Organization & Coherence— several paragraphs developed in depth; 

several ideas or examples explored with specific details; essay uses a 
variety of transitional words to link thoughts together. Longer and deeper 
than a 5 essay, with more sophisticated sentence structures. 
Mechanics— may contain some surface errors which are easily editable. 

7 Organization & Coherence— fluent, idiomatic, capable of English 28 

work; essay may still have minor mistakes, but exhibits greater depth 
and/or greater length than a 6. 
Mechanics— fewer surface errors than a 6. 

8 Organization k Coherence- ready for English 101; essay is persuasive, 

poised, and substantial, elaborating its points with grace and style. 
Mechanics— control over almost all grammatical conventions. 
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Appendix A. 2. Data on Standardized English Proficiency Tests 

Evaluated by the ESL Committee Page 1 of 2 pages 



Commercial ESL Tests Reviewed for Use in LACCD 
ESL Placement Procedure 



Off-the-shelf Tests 

Basic English Skills Test, Literacy (BEST-L) 
Center for Applied Linguistics, Washington, DC 
(45 minutes) 

Comprehensive English Language Test (CELT) 
McGraw-Hill Book Company, NY 
(2 hours, 5 mins) 

English-second-language Placement Test (EPT) 
ESL Teachers' Resource Instructor, San Francisco 
(30 minutes) 

Michigan English Placement Test (MEPT) 
University of Michigan, Ann Arbor, MI 
(1 hour, 15 mins) 



Secondary Level English Proficiency Test (SLEP) 
Educational Testing Service, Princeton, NJ 
(1 hour, 30 mins) 

Structure Tests - English Language (STEL) 
Harper & Row Publishers, Scranton, PA 
(30 minutes) 

Test of English Proficiency Level (TEPL) 
Alemany Press, Hayward, CA 
(60 minutes) 



Secure Tests 

Preliminary Test of English as a Foreign 

Language (Pre— TOEFL) 
Educational Testing Service, Princeton, NJ 

(1 hour, 15 minutes) 



General Tests of English Language 

Proficiency (G-TELP) , Levels 3, 2, 1 

San Diego State University, San Diego, CA 
(Level 3, 90 minutes; Level 2, 110 minutes; 
Level 1, 105 minutes) 



Areas Tested 

reading 
writing 



structure 
listening 
vocabulary 

structure 



structure 
listening 
vocabulary 
reading 

listening 
reading 



structure 



structure 

reading 

writing 



structure & 
written expression 
listening 
vocabulary & 
reading 



grammar 

listening 

reading 
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ESL Test Breakdown by Skill Areas 



4 of Items 



Testing Time 



Comments 



STRUCTURE 
CELT 
MEPT 
EFT 

STEli 



TEPL 



75 
30 
50 

50 

88 



45 mins 
* 

30 mins 
30 mins 
** 



(3 options/ item; 
4 forms) 

( 3 levels ; 
2 forms each) 

(5 options/ item; 
items embedded in 
reading passages) 



CELT 



MEPT 



SLEP 



50 



20 



75 



45 mins 



25 mins 



45 mins 



20 gs, 20 sts, 
10 dialogues 

3 options / item ; 
gs , sts 

25 pics, 2 0 sts, 
12 map, 18 conv 



READING 
MEPT 
SLEP 

TEPL 



20 
70 



45 mins 



(sen-level only) 

(27 based on pics; 
43 rding comp, 
gr, voc based on 
passages) 



VOCABULARY 
CELT 
MEPT 



75 
30 



35 mins 



(35 w/blank in 

(all w/blank in 
sen) 



* 50 minutes t.o complete all three sections (str, rding, voc) 
** 60 minutes to complete all sections (5 tasks) ; speeded test 



Page 2 of 2 pages 
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Appendix A. 3. Illustrative SLEP Items: Listening Comprehension 



■ ■ ■ Sample Questions 
Section l 

The first section of the SLEP test measures ability to understand spoken 
English and is 35-40 minutes long. It is droded into four parts, with fewr different 
types of questions. 

PtrtA 

For the first type of question, die student must match one of four recorded 
sentences with a picture in the test book The sentences are spoken only cocr and 
are not printed in the test book. This part contains items dealing with correct 
recognition of minimal pair contrasts, juncture, sties*, sound dusters, tense, voice, 
prepositions, and vocabulary. 

Stmtpie Questions 

Note: Rcniresarefbrillusaaavepurpoaaoniy. 
Actual pictures and drawings in the test book' 
let are two to four times larger than sample 
pictures in this brochure. 

1. On tape: 

Look at the picture marked 1 . 

On tape: 

(A) There a an <mw> n the sic?. 

(B) The building has a tail touw. 

(C) The jud&iskowmghhhead. 

(D) There a a toy m front of the buiidmg. 



Z. On tape: 



Look at the picture markd 2. 
On tape: 

(A) Thx bird is standing an tap of thipoit. 

(B) The bird is /lying over the fence. 

(C) The bird is digging m the sand. 

(D) The bird is eating the grass. 
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Appendix A . 3 . , 



con't: Listening Comprehension 



3. On tape: 

Look at the picture marked 3 . 

On rape: 

(A) There's a statue of a ikm. 

(B) The line a very sorsgfu. 

(C) The ume a near the uwdotu. 

(D) There's a lane near the building. 




On tape: 

Looka thepkaatTncaked^. 
On tape: 

(A) The brain is proceaed by bone. 

(B) The tram is on me track. 

(C) The dnsiis stopped up. 

(D) The rain is conwig down. 




PartB 

These questions approximate the type of dictation exercises used frequently 
in English language classes: the student must match a sentence printed in the test 
book with a sentence heard on the tape. The questions focus on the relationship 
between structure and meaning. 

Sample Questions 

1. On tape: The ckitt can/mish k m less man an hour. 

In test book: (A) The classes can't finish in half an hour. 

(B) The class won t be finished for an hour. 

(C) The classes will take at least an hour. 

(D) The class can finish it in less than an hour. 

2. On rape: Why aren't me-? /bong me car? 

In test book: (A) Are they fixing the car? 

(B) I'm fixing the car. 

(C) Why aren't they fixing the car? 

(D) The car has been fixed. 

3. On tape: While I u*b \usiang for my sister, she game news. 

In test book; (A) While I was waiting for my sister, she got the news. 

(B) While my sister was waiting for me, she got the news. 

(C) 1 was waiting for my sister to get the news. 

(D) I was waiting for my sister when 1 got the news. 
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Xppandix A, con*t: illustrative 8L8P It«M 



P«rtC -~ 

For the second cype of quemon. ifac ttudene torn toatnapmaieteKbook 
( see paw 1 1 )• Saeeaand buddings on the nap am labeled, and daw ate faurcm. 
mariced A, B, C and D. The sojdenr oust choose the one car that istnewaceof 
a b«rf convcnwoo w the tmadin* 
U ng «^culii^axidptagnwiccDO^ 
of bukfang nana andaaoaated vceabt^daance,andoffle. 

SoMpkOueaami 



On tapes 
(asm) 



(diiid 



The museum Kas a special edvibtt this week. Why don't 

I'd like to very much. If we connnue on Mackerel to me cnde end |0 

jnwi to Saimcn, we an park on Cod Lent. 

VGfock ear an? die people in? 



2 On apet 

guhctnt. 

) Ui not bo bvxd. If u* near n^t wo Ban and men ga wuh on 

Salmon, w* «u£ end up at die eirde. 
) «T*m car are die people*? 



(< 

(thud 

3. Onapet 

(woman) TV^dteiort jowfioteirauerf i 

a the count. 

(man) Thorto|oodid«a.rBpnoitno^n^ 

Avenue. We con pant « die lot across die meet from die awn. 

(thiid voice) Wfcokcsrani die people W 



today. Lee's sop 



4. On apet He d&'t Jew now so set to die pm- 

IntcMDOoks (A) He didn't to to the tjym. 

(B) Hcaplainedhowcoiaethefym. 

(C) He told is id set d the cym. 

(D) He didn't knew how id pa aa the gym- 



S. 



Ontaptt D^hosonebnxhcrofidonesm.andjodoBjone. 
Mtbaoks (A)BiUhacmbtothefandaneiUKe&andsodoet)ar«. 

(B) Bill hat one b l ather and a nsaar named jane. 

(C) Bill and Jane a« crodwr and won. 

(D) Bill's btother and tuoat like to be with Jams. 
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Appendix A.3., con't: Listening Comprehension 




Appendix A. 3., Listening Comprehension, concluded 



Part D 

The questions in this pare are based on conversations, recorded by Ameri- 
can high school students, that r e prese nt typical secondary school situations. The 
conversations take place in various pans ot a school and deal with events that 
typically occur in each location. Conversations also deal with extracurricular 
activities, academic subjects, school closings, and holidays. For each recorded 
question, the student must choose one of tour answers printed in the test book. 



Sample Questions 

1. On tape; 

(Bob) I heard, that it is supposed to be a very good bard. Since me gxme 
jams at 7:30, Nancy, I'll pick you up a 7. 
(Nancy) That's fine. I'll be ready. It takes 15 minutes to get to me gym, so 
we'll hauenme. 
(third voice) At what nme will they amue at the gym? 

In test book: (A) 6:45. 

(B) 7:00. 

(C) 7:15. 

(D) 7:30. 
For questions 2 and 3. 

2. Oti ape? 

(Nancy) lane, what are you going to wear to the game? 
(Jane) I'm noc sure ya. I don't want to have a heavy sweater on at we 
dance. It'll be prory warm m the gym. I'll probabry wear a light 
■ire J, even though the weather ovicide might not be so warm, 
'third voice) What is the grri going to wear? 

In test book: (A) A heaw sweater. 

• B) A heavy coat. 

(C) Some light slacks. 

(D) A light dress. 

3. (On tape) What is the girl's reason for this decision? 

In test book: (A) She expects it to be cold outside. 

(B) She expects it to be warm inside. 

(C) It is going to snow. 
iD) It will be verv windv. 
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Appendix A. 3, Reading Comprehension 



Section 2 



The second section of the test is 40 minutes long and measures ability no 
understand Written English. The questions cover grammar, vocabulary, and 
reading comprehension. There arc three parts id Section 2. 





SampU Questions 

1 . All those wet clothes. The children will want to sav outside and 111 spend my 
time trying to keep them dry. 

2. I can haidlv wait to make the first snowball. I've been waiting ail year to get 
back at her. 

3. Oh. mv aching back. The car wiil be covered and I'll have to shovel ic ouc 

4. Isn't it great that school might be closed? I'd much rather have run outside man 
stay in school. What better way to spend a snowy dav. 

5. I'm going to be awfully hungry. 1 shouldn't have hidden that bone, it would 
have been better to leave it in the house. 



PartB 



For the questions in this part, the student must match a pnnred sentence 
with one of four drawings. The particular focus at this item cvpe is the use or" 
prepositions, pronouns, adverbs, ana numbers. 
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Appendix A. 3, con't: Reading Comprehension 



Sample Questions 

1. One giri is eanng ice cream but two aren't. 



i > 



v. 



2. The small square is in the upper left comer. 



3. He is bending over to pick up the box. 



7=^ 

7? 



4. The car almost hit him while he was crossing the street. 



rv 



^ "r\ - -J I TV 
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Appendix A. 3, con't: Reading Comprehension 



PartC 

This part of" Section 2 contains questions of two types. In one, the student 
must complete passages by selecting the approphatc words or phrases from among 
four choices printed at intervals in the passages. 

Sample Passage and Questians 



1. Sound is something we 



(A) hears. 

(B) hearing. 

(C) heard. 

(D) hear. 



It comes to your 



(A) eyes 

(B) nose 

(C) ears 

(D) mouth 



in different ways. It might be pleasant. 



3. like the voice of a friend. 



4. of a train's wheels or> a railroad 




(A) station. 

(B) track. 

(C) light 

(D) conductor. 



unpleasant, like the screech 



Some sounds arc loud, 



5. and some aie soft; some are high, and some are 



(A) full. 

(B) low. 

(C) quiet. 

(D) big. 



Sound is 



6. very 



(A) importance 

(B) importantly 

(C) important 

(D) import 



to us because it is the basic means of communicanon. 



In the second type of question, the student must answer questions about the 
passage for which he or she supplied the missing words or phrases. 

Sample Questions 

7. What does screech in line 3 mean? 

(A) noise (B) mooon (C) place (D) piece 
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Appeidix A. 3., Reading Comprehension, concluded 



8. Which of the phrases below is another example cr a pleasant sound similar to 
the phraae in the sentence that begins in line 2, like the voice of amend"? 

(A) Like the ring of an alarm (B) Like the wail of a siren 
(C) Like the honk of a horn (D) Like the song of a bird 

9. Which sentence below has almost the same meaning as the sentenc char 
begin* in line 5? 

(A) It is meaningful to communicate with sound. 

(B) The main way vw communicate is with sound. 

(C) The meaning of sound is base cd communication. 

(D) In order to communicate, wc need basic sounds. 



In this pare of Secnon 2, die student must read a short literary passage and 
answer questions about ic 

Sample Passage and Questions 

The footsteps began about a quarter past one o'clock in the morning, 
a rhythmic, quick-cadenced walking around the dining room table. My 
mother was asleep in one room upstairs, my brother Herman in another, 
grandfather was in the acne, in the old walnut bed. I had just stepped out of 
the bathtub and was busily rubbing myself with a towel when I heard the 
steps. They were the steps of a man walking rapidlv around the dining room 
cable downstairs. 

1. What did the wnrer hear? 

.(A) A soldier marching (ft) His brother snonng 
fC) His motner taiking ;D) A person walking 

2. Where did the sounds come from.' 

(A) Theatnc (B) The dining room 
(C) The bathroom (D) The stairs 



Part D 



3. What was most of the family doing? 

(A) Listening (B) Working (C) Bathing (D) Sleeping 
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Appendix B.1a. Distribution of Scores on Variables Used in Placement, 
for All LACCD-Students Assessed: Spring 1991 through Spring 1992 



•S'.cP LC (Short version) 



Cum 



Cum 



Cum 



Score 


Freq Pet Pet 


Score 


Freq Pet Pet 


Score 


Freq Pet 


Pet 


1 


17 


0 


0 


16 


190 


2 


12 


31 


301 


3 


46 


2 


36 


0 


1 


17 


179 


2 


14 


32 


362 


3 


50 


3 


27 


0 


1 


18 


210 


2 


16 


33 


420 


4 


54 


4 


40 


0 


1 


19 


161 


2 


17 


34 


464 


4 


58 


5 


37 


0 


1 


20 


192 


2 


19 


35 


484 


5 


63 


6 


46 


0 


2 


21 


197 


2 


21 


36 


505 


5 


67 


7 


51 


0 


2 


22 


221 


2 


23 


37 


512 


5 


72 


8 


62 




3 


23 


226 


2 


25 


38 


521 


5 


77 


9 


70 




4 


24 


202 


2 


27 


39 


557 


5 


82 


10 


77 




4 


25 


231 


2 


29 


40 


470 


4 


87 


11 


102 




5 


26 


260 


2 


31 


41 


415 


4 


91 


12 


110 




6 


27 


265 


3 


34 


42 


355 


3 


94 


13 


120 




8 


28 


292 


3 


37 


43 


299 


3 


97 


14 


112 




9 


29 


349 


3 


40 


44 


247 


2 


99 


15 


158 




10 


30 


351 


3 


43 


45 


84 


1 


100 




lid cases 




10587 



















SLEP RC (Short version) 



Cum 

Score Freq Pet Pet 



Scor" 



Cum 

Freq Pet Pet 



Cum 

Score Freq Pet Pet 



1 


24 


n 


0 


20 


302 


3 


30 


39 


233 


2 


25 


0 


0 


21 


375 


4 


33 


40 


196 


3 


22 


0 


1 


22 


377 


4 


37 


41 


223 


4 


53 


1 


1 


23 


327 


3 


40 


42 


172 


5 


31 


0 


1 


24 


380 


4 


43 


43 


176 


6 


69 




2 


25 


370 


3 


47 


44 


177 


7 


103 




3 


26 


337 


3 


50 


45 


140 


8 


78 




4 


27 


340 


3 


53 


46 


95 


9 


137 




5 


28 


328 


3 


56 


47 


83 


10 


123 




6 


29 


317 


3 


59 


48 


79 


11 


139 




8 


30 


344 


3 


63 


49 


66 


12 


184 


2 


9 


31 


312 


3 


66 


50 


53 


13 


186 


2 


11 


32 


295 


3 


68 


51 


31 


14 


229 


2 


13 


33 


261 


2 


71 


52 


35 


15 


278 


3 


16 


34 


260 


2 


73 


53 


19 


16 


271 


3 


18 


35 


232 


2 


75 


54 


8 


17 


289 


3 


21 


36 


309 


3 


78 


55 


5 


18 


267 


3 


24 


37 


271 


3 


81 






19 


323 


3 


27 


38 


228 


2 


83 






Valid 


case'. 




10587 















85 
87 
89 
91 
93 
94 
96 
96 



1 97 
1 98 
1 99 



99 
99 



0 100 
0 100 
0 100 
0 100 
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Total SLEP Score (Short version) 
Cum Cum Cum 

Value Frea °ct Pet Value Freq Pet Pet Value Freq Pet Pet 



3 


10 


0 


0 


36 


113 


1 


16 


69 


175 


2 


71 


4 


9 


0 


0 


37 


146 


1 


17 


70 


216 


2 


73 


5 


12 


0 


0 


38 


137 


1 


19 


71 


173 


2 


74 


6 


8 


0 


0 


39 


127 


1 


20 


72 


203 


2 


76 


7 


6 


0 


0 


40 


103 


1 


21 


73 


172 


2 


78 


8 


5 


0 


0 


41 


141 


1 


22 


74 


187 


2 


80 


9 


20 


0 


1 


42 


143 


1 


24 


75 


181 


2 


81 


10 


26 


0 


1 


43 


138 


1 


25 


76 


181 


2 


83 


11 


30 


0 


1 


44 


168 


2 


27 


77 


158 


1 


84 


12 


22 


0 


1 


45 


149 


1 


28 


78 


166 


2 


86 


13 


27 


0 


2 


46 


1 29 


1 


29 


79 


131 


1 


87 


14 


22 


0 


2 


47 


114 


1 


30 


80 


144 


1 


89 


15 


23 


0 


2 


48 


159 


2 


32 


81 


143 


1 


90 


16 


37 


0 


2 


49 


177 


2 


33 


82 


112 


1 


91 


17 


15 


0 


3 


50 


185 


2 


35 


83 


131 


1 


92 


18 


36 


0 


3 


51 


194 


2 


37 


84 


126 


1 


93 


19 


44 


Q 


3 


52 


168 


2 


39 


85 


124 


1 


95 


20 


55 


1 


4 


53 


157 


1 


40 


86 


77 


1 


95 


21 


40 


0 


4 


54 


188 


2 


42 


87 


75 


1 


96 


22 


34 


0 


5 


55 


176 


2 


43 


88 


73 


1 


97 


23 


75 




5 


56 


189 


2 


45 


89 


7a 


1 


97 


24 


63 




6 


57 


186 


2 


47 


90 


68 


1 


98 


25 


76 




7 


58 


213 


2 


49 


91 


41 


0 


98 


26 


72 




7 


59 


208 


2 


51 


92 


48 


0 


99 


27 


60 




8 


60 


205 


2 


53 


93 


20 


0 


99 


28 


78 




9 


61 


197 


2 


55 


94 


28 


0 


99 


29 


88 




9 


62 


205 


2 


57 


95 


12 


0 


100 


30 


89 




10 


63 


200 


2 


59 


96 


23 


0 


100 


31 


93 




11 


64 


223 


2 


61 


97 


21 


0 


100 


32 


108 




12 


65 


217 


2 


63 


98 


2 


0 


100 


33 


113 




13 


66 


240 


2 


65 


99 


3 


0 


100 


34 


108 




14 


67 


200 


2 


67 


100 


3 


0 


100 


35 


88 




15 


68 


205 


2 


69 










Valid 


cases 




10587 



















Essa. rating 

Valid Cum 
Rating Frequency Percent Percent Percent 



0 


175 


1.7 


1.7 


1.7 


1 


1553 


14.7 


14.7 


16.3 


2 


2173 


20.5 


20.5 


36.8 


3 


2384 


22.5 


22.5 


59.4 


4 


25'".0 


23.8 


23.8 


83.2 


5 


1207 


11.4 


11.4 


94.6 


6 


466 


4.4 


4.4 


99.0 


7 


95 


.9 


.9 


99.9 


8 


14 


.1 


.1 


100.0 


Total 


10587 


100.0 


100.0 





Valid cases 10587 
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Appendix B.2. Descriptive Statistics for Placement Variables, 
by College and ESL Level (Last Course Enrolled) 

Shortened SLEP Essay Composite 

College ESL N LC LCSD RC RCSD SLEP SLSD ESSAY ESSD COHP TOSD 
Level H Sd H Sd H Sd H Sd H Sd 



1 


944 


17 


9 


14 


7 


32 


15 


1.9 


1.2 


29 


13 


2 


613 


21 


7 


18 


6 


39 


10 


2.4 


1.1 


36 


9 


3 


554 


27 


7 


22 


6 


49 


9 


3.1 


1.0 


46 


8 


4 


736 


32 


5 


27 


6 


59 


10 


3.7 


.9 


55 


8 


5 


619 


36 


5 


32 


6 


68 


10 


4.4 


.9 


64 


8 


6 


331 


38 


6 


37 


9 


75 


13 


4.9 


1.1 


71 


12 


7 


75 


36 


5 


34 


9 


70 


12 


4.8 


1.1 


67 


11 



East 


> 


71 


28 


8 


21 


7 


49 


12 


1.6 


.7 


33 


8 


2 




362 


34 


6 


28 


8 


62 


13 


2.2 


.7 


42 


8 


2 


5 


163 


38 


4 


35 


7 


73 


10 


2.9 


.8 


51 


8 


2 


6 


134 


38 


6 


38 


9 


76 


13 


3.3 


1.1 


56 


12 


2 


7 


212 


33 


8 


31 


11 


64 


17 


2.4 


.9 


44 


12 



Harbor 


2 


100 


33 


7 


30 


9 


62 


15 


2.4 


1.2 


52 


13 


3 


4 


96 


32 


8 


27 


8 


58 


14 


2.3 


1.1 


49 


11 


3 


5 


156 


37 


6 


36 


8 


73 


13 


3.1 


1.0 


62 


10 


3 


6 


49 


38 


6 


37 


8 


75 


13 


3.8 


1.3 


67 


12 


3 


7 


30 


38 


5 


37 


8 


75 


11 


3.3 


1.3 


64 


11 



Mission 


1 


361 


16 


9 


16 


8 


33 


14 


1.1 


.4 


24 


12 


4 


2 


217 


23 


9 


22 


9 


45 


16 


1.9 


.7 


38 


12 


4 


3 


361 


28 


8 


26 


8 


53 


13 


2.7 


.9 


48 


8 


4 


4 


374 


32 


7 


31 


8 


63 


12 


3.4 


.8 


56 


9 


4 


5 


281 


36 


5 


37 


7 


73 


10 


3.9 


1.1 


66 


7 


4 


6 


123 


38 


6 


41 


8 


79 


13 


4.2 


1.4 


71 


12 



Pierce 


4 


138 


39 


5 


36 


8 


75 


11 


4.1 


.5 


61 


6 


5 


5 


127 


40 


4 


39 


8 


79 


10 


4.7 


.6 


67 


7 


5 


6 


113 


42 


3 


44 


7 


85 


9 


6.0 


.9 


80 


9 


5 


7 


115 


35 


7 


32 


10 


68 


16 


3.9 


1.6 


57 


17 


Trade 


1 


28 


26 


6 


19 


7 


45 


11 


2.8 


1.6 


42 


12 


6 


2 


82 


27 


8 


19 


7 


47 


12 


4.0 


1.4 


48 


11 


6 


3 


170 


27 


7 


20 


7 


48 


12 


4.3 


1.3 


51 


11 


6 


4 


143 


33 


6 


25 


7 


57 


11 


4.9 


1.1 


60 


8 


6 


5 


54 


36 


3 


32 


7 


68 


7 


5.6 


1.4 


70 


7 



Valley 


4 


935 


32 


7 


25 


8 


57 


12 


2.2 


.9 


48 


9 


7 


5 


267 


38 


5 


35 


7 


73 


11 


3.2 


1.0 


62 


9 


7 


6 


206 


40 


5 


40 


9 


80 


11 


4.1 


1.3 


71 


10 


7 


7 


208 


39 


4 


38 


8 


77 


11 


3.9 


1.2 


68 


10 


Uest 


3 


510 


2- 


8 


20 


7 


42 


14 


2.9 


1.0 


38 


9 


8 


4 


358 


35 


6 


28 


8 


62 


10 


3.9 


.9 


52 


10 


8 


5 


257 


38 


4 


33 


8 


71 


9 


4.4 


1.0 


62 


9 


8 


6 


116 


40 


3 


41 


7 


82 


6 


5.2 


1.0 


71 


10 



Note. All levels not represented at all colleges. 
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Appendix B.3. Mean Score on Placement Variables, by Language Group 
and College: All Students Assessed 

Shortened SLEP 



LANG 


College N 


LC 


RC 


SLEP 


ESSAY 


COMP 




(code) 




























English 




1 


City 1 


41 


25.7 


22.7 


50.1 


3.4 


45.4 


City 


1 


East 2 


30 


33.4 


29.0 


62.5 


2.2 


41.6 


East 


1 


Harb 3 


8 


39.6 


35.8 


75.4 


2.6 


61.8 


Harbor 


1 


Hiss 4 


30 


32.9 


33.9 


66.9 


3.4 


59.1 


Mission 


1 


Pier 5 


21 


40.2 


40.5 


80.7 


4.7 


67.6 


Pierce 


1 


Trad 6 


8 


24.5 


22.8 


47.3 


3.3 


45.1 


Trade 


1 


Vail '7 


38 


34.5 


34.2 


63.7 


3.0 


58.6 


Valley 


1 


West 8 


23 


39.4 


34.1 


77.1 


3.8 


57.1 


West 














Armenian 




2 


1 


1019 


23.1 


18.1 


41.3 


2.5 


38.3 




2 


2 


16 


33.3 


25.8 


59.1 


2.4 


42.7 




2 


4 


12 


28.0 


25.0 


53.0 


2.3 


45.4 




2 


5 


18 


40.1 


35.9 


76.0 


4.5 


64.7 




2 


7 


154 


32.7 


23.1 


55.8 


2.6 


48.7 




2 


8 


4 


12.0 


20.3 


18.0 


2.5 


35.0 





Chinese 



3 


1 


97 


29.8 


27.9 


58.5 


3.8 


54.4 


3 


2 


369 


34.8 


31.1 


65.9 


2.o 


46.1 


3 


3 


34 


35.3 


35.8 


71.1 


3.3 


61.4 


3 


4 


5 


39.2 


28.4 


67.6 


4.6 


65.4 


3 


5 


37 


36.5 


34.3 


70.8 


4.2 


59.9 


3 


6 


11 


33.8 


23.5 


61.0 


3.9 


50.9 


3 


7 


33 


38.5 


33.9 


72.4 


3.3 


62.8 


3 


8 


33 


31.7 


26.9 


58.0 


3.7 


49.7 



Farsi 



4 


1 


21 


29.4 


24.4 


53.8 


3.1 


49.5 


4 


2 


2 


38.0 


18.5 


56.5 


2.5 


42.5 


4 


3 


3 


35.0 


27.7 


62.7 


3.7 


57.7 


4 


4 


3 


41.7 


37.7 


79.3 


3.7 


68.7 


4 


5 


62 


39.5 


34.4 


73.9 


4.2 


61.4 


4 


6 


1 


28.0 


10.0 


38.0 


4.0 


44.0 


4 


7 


103 


35.0 


26.2 


61.2 


2.5 


52.1 


4 


8 


73 


33.7 


25.4 


54.8 


3.6 


48.8 



Filipino 

5 1 34 36.6 ».1 72.7 4.5 67.4 

5 2 2 39.0 24.5 63.5 2.5 45.0 

5 3 20 38.3 38.4 76.7 2.8 62.8 

5 4 1 44.0 52.0 96.0 5.0 85.0 

5 5 4 42.3 47.5 89.8 5.5 77.3 

5 7 22 39.4 38.7 78.1 3.5 67.2 

5 8 3 43.0 27.7 60.0 4.7 61.0 
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LANG College N 
(code) 



Shortened SLEP 
LC RC SLEP 



ESSAY COMP 



Japanese 



6 


1 


98 


34.7 


30.8 


65.5 


4.3 


62.3 


6 


2 


31 


36.4 


32.0 


tS.4 


2.6 


47.3 


6 


3 


81 


38.1 


35.9 


74.0 


3.3 


63.4 


6 


4 


18 


33.7 


33.5 


67.2 


3.7 


60.4 


6 


5 


37 


40.2 


38.4 


78.6 


4.9 


68.9 


6 


6 


1 


41.0 


29.0 


70.0 


6.0 


73.0 


6 


7 


121 


36.8 


32.0 


68.9 


3.2 


60.0 


6 


8 


225 


36.0 


30.0 


65.6 


4.1 


55.7 
















Xorean 


7 


1 


574 


32.8 


29.3 


62.1 


3.6 


56.8 


7 


2 


15 


36.8 


32.8 


69.6 


2.5 


46.9 


7 


3 


17 


37.9 


33.9 


71.9 


3.0 


60.9 


7 


4 


21 


34.7 


31.0 


65.7 


3.4 


58.6 


7 


5 


39 


38.8 


37.9 


76.7 


4.7 


66.3 


7 


6 


25 


34.6 


24.6 


59.2 


4.1 


57.1 


7 


7 


98 


35.7 


32.4 


68.1 


2.7 


57.1 


7 
I 


O 
O 


77 


T*> X 
jj * J 


97 ^ 


AA 1 


4.0 


53.1 














Russian 


8 


1 


166 


27.0 


22.4 


49.4 


3.4 


47.5 


8 


2 


23 


31.7 


24.2 


55.9 


1.9 


37.3 


8 


4 


7 


23.9 


24.7 


48.6 


4.0 


49.6 


8 


5 


11 


40.6 


40.5 


81.1 


4.8 


68.9 


8 


6 


1 


40.0 


39.0 


79.0 


4.0 


68.0 


8 


7 


124 


37.1 


29.4 


66.5 


3.0 


57.6 


8 


8 


28 


38.0 


26.6 


59.0 


4.4 


54.1 
















Spanish 


9 


1 


1499 


24.8 


23.1 


48.0 


3.1 


45.3 


9 




336 


34.4 


31.5 


66.0 


2.4 


45.1 


9 


3 


153 


31.1 


29.4 


60.8 


2.5 


:o.8 


9 


4 


1629 


26.3 


26.9 


54.0 


2.6 


45.5 


9 


5 


83 


39.4 


38.8 


78.2 


4.7 


67.1 


9 


6 


495 


28.5 


21.2 


49.7 


4.0 


50.5 


9 


7 


564 


34.4 


30.7 


65.1 


2.8 


55.4 


9 


8 


500 


29.6 


25.0 


54.5 


3.5 


46.7 



Vietnamese 



10 


1 


55 


28.5 


24.3 


52.8 


4.0 


52.4 


10 


2 


70 


30.4 


27.7 


58.2 


2.4 


41.9 


10 


3 


5 


28.6 


25.4 


54.0 


2.2 


45.8 


10 


4 


7 


36.4 


30.6 


78.8 


4.7 


57.6 


10 


5 


69 


34.5 


32.4 


66.9 


4.2 


58.4 


10 


6 


20 


22.1 


18.0 


40.1 


4.5 


46.7 


10 


7 


75 


30.5 


25.6 


56.1 


2.7 


49.5 


10 


8 


9 




25.8 




4.8 


54.7 
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Shortened SLEP 
LANG College N LC RC SLEP ESSAY COHP 
(code) 













Other 


Language 


1 


263 


31.3 


26.8 


58.4 


3.8 


54.4 


2 


38 


37.9 


32.0 


69.9 


2.6 


48.3 


3 


27 


38.2 


35.4 


73.6 


2.6 


60.4 


4 


47 


30.9 


35.3 


66.8 


3.6 


58.7 


5 


111 


41.0 


40.1 


81.0 


4.9 


69.9 


6 


12 


31.3 


21.8 


53.0 


4.4 


54.6 


7 


273 


36.2 


31.0 


67.1 


2.9 


57.3 


8 


259 


31.7 


27.0 


59.3 


3.9 


50.9 



Lang Hissing 



0 


1 


101 


29.9 


28.7 


58.6 


3.7 


54.9 


0 


2 


20 


39.0 


38.0 


77.0 


3.0 


53.3 


0 


3 


168 


30.2 


27.1 


57.3 


2.5 


49.2 


0 


4 


272 


27.8 


27.3 


55.4 


2.6 


48.3 


0 


5 


22 


42.0 


41.7 


83.6 


5.0 


71.2 


0 


6 


79 


30.0 


20.7 


51.0 


4.3 


52.8 


0 


7 


36 


33.5 


33.0 


66.5 


2.8 


56.3 


0 


8 


40 


34.4 


27.1 


63.6 


2.9 


50.4 



Number of cases read = 92 
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Appendix B.4. Relationship Between Placement Variables and Selected 
Demographic Variables, By College 



Correlation with designated demographic variable 



Col lege/ 




Shortened SLEP 






demographic 


LC 


RC 


SLEP 


ESSAY 


TOTAL 


City 

GENDER 

AGE 

EDLEVEL 


- .20** 
-.28** 
.13** 


-.20** 
-.23** 
.14** 


-.22** 
- .26** 
.14** 


-.13** 
- . 16** 
.16** 


-.19** 
-.27** 
.16** 


East 

GENDER 

AGE 

EDLEVEL 


-.13** 
-.08* 
.14** 


-.13** 
.04 
.18** 


-.14** 
-.01 
.18** 


.04 
.02 
.11** 


-.05 
-.00 
.16** 


Harbor 
GENDER 
AGE 

EDLEVEL 


-.12** 
-.26** 
.29** 


-.09* 
-.20** 
.33** 


-.10* 
-.23** 
.33** 


-.08 
-.10* 
.16** 


-.12** 
-.24** 
.31** 


Mission 

GENDER 

AGE 

EDLEVEL 


-.09** 
-.19** 
.21** 


-.08** 
-.03 
.26** 


-.09** 
-.12** 
.25** 


- .04 
-.02 
.2.1** 


-.09** 
-.11** 
.25** 


Pierce 
GENDER 
AGE 

EDLEVEL 


.00 
-.10* 
.08 


-.02 
-.05 
.09* 


-.01 
-.08 
.09* 


.01 
-.06 
.04 


.01 
-.07 
.07 


Trade 

GENDER 

AGE 

EDLEVEL 


-.05 
-.19** 
.15** 


.01 
-.11** 
.22** 


-.03 
-.15** 
.21** 


-.05 
-.11** 
.16** 


-.04 
-.19** 
.22** 


Valley 
GENDER 
AGE 

cni cv/ci 
tULcVcL 


-.16** 
-.21** 


-.16** 
-.09** 
.11** 


-.18** 
-.16** 
.11** 


-.02 
-.13** 
.15** 


-.14** 
-.18** 
.14** 


West 

GENDER 

AGE 

EDLEVEL 


-.17** 
-.23** 
.18** 


-.11** 
-.09** 
.12** 


-.22** 
-.20** 
.16** 


.00 
.00 
.26** 


-.07** 
-.06* 
.18** 


Total Sample 

GENDER 

AGE 

EDLEVEL 


-.13** 
-.25** 
.20** 


-.13** 
-.15** 
.19** 


-.14** 
-.21** 
.20** 


-.06** 
-.10** 
.18** 


-.12** 
-.19** 
.22** 


* - Signif. 


LE .05 


** - Signif. LE .01 


(2-tailed) 



Note. Correlation coefficients reflecting relationships between 
placement variables, on the one hand, and selected demographic variables, 
indicate generally that (a) females tend to earn lower scores on the 
placement variables than do males, (b) older students tend to learn lower 
scores than to younger students, and (c) students reporting more formal 
education tend to perform better than do students with less formal 
education. The general negative relationship between age and performance 
on the placement tests is stronger for listening comprehension than for 
either reading comprehension or writing ability (essay rating). 

Page 1 of 1 pages 



LevelExp.Tbl 85 

BEST COPY AVAILABLE 

o 107 

ERIC 



Appendix B.5. Results of Regression of Essay Rating on SI EP 
IX and RC in the AU-LACCD Sample: All Students 
Assessed, All Colleges (Basis for Residual Analysis, 
Section III) 



Mean Std Dev Label 



ESSAY 

IX 

RC 

SLEP 
TOTAL 



3.092 
30.195 
26.986 
57.181 
50.^10 



1.483 
9.912 
10.913 
19.335 
16.770 



N of Cases = 10,587 
Correlation: 





ESSAY 


LC 


RC 


SLEP 


TOTAL 


ESSAY 


1.000 


.486 


.491 


.526 


.788 


LC 


.486 


1.000 


.723 


.921 


.851 


RC 


.491 


.723 


1.000 


.935 


.844 


SLEP 


.526 


.921 


.935 


1.000 


.913 


TOTAL 


.788 


.851 


.844 


.913 


1.000 


Equation Number 1 


Dependent Variable.. 


ESSAY 





Variable(s) Entered on Step Number 
1.. RC 
2.. LC 

Multiple R .52625 

R Square .27694 

Adjusted R Square .27680 

Standard Error 1.26116 



Analysis of Variance 

DF 

Regression 2 
Residual 10584 



Sun of Squares 
6447.54537 
16834.10949 



Hean Square 
3223.77268 
1.59052 



F = 2026.86160 Signif F = .0000 
Variables in the Equation 



Variable 

RC 
LC 

(Constant) 



B 



.0399 
.0409 
.7812 



SE B 

.001627 
.001791 
.039589 



Beta 



T Sig T 



.293 
.273 



24.515 
22.833 
19.733 



.0000 
.0000 
.0000 



Residuals Statistics: 



Hin 


Max 


Mean 


Std Dev 


N 


♦Predicted Essay .9019 


4.8152 


3.0924 


.7804 


10587 


♦Residual -4.2537 


4.9499 


.0000 


1.2610 


10587 


*Z- sea led pred -2.8068 


2.2075 


.0000 


1.0000 


10587 


*Z-scaled res -3J3729 


3.9249 


.0000 


.9999 


10587 



Total Cases = 10,587 
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Table G.I. Descriptive Statistics for Placement Variables in Unit Samples 
(Sfjdents Classified by Course and Term ["SAMP"]): Parameters Used 
for Z-Scaled Transformation of Basic Test Variables 



COL LEV SUBJ COUR SAMP 



Ci 



1 


263 


61 


1 


1 


263 


71 S91 


1 


1 


263 


71 F91 


2 


1 


263 


71 S92 


3 


1 


263 


74 


1 


1 


263 


74 


2 


1 


263 


74 


3 


1 


263 


75 


1 


1 


263 


75 


2 


1 


263 


75 


3 


2 


263 


72 


1 


2 


263 


72 


2 


2 


263 


72 


3 


2 


263 


76 


1 


2 


263 


76 


2 


2 


263 


76 


3 


2 


263 


78 


1 


2 


v j3 


78 


2 


2 


363 


78 


3 


3 


263 


63 


3 


3 


263 


73 


1 


3 


263 


73 


2 


3 


263 


73 


3 


3 


263 


77 


1 


3 


263 


77 


2 


3 


263 


77 


3 


3 


263 


79 


1 


3 


263 


79 


2 


3 


263 


79 


3 


4 


399 


80 


1 


4 


399 


83 


1 


4 


399 


84 


1 


4 


39 


84 


2 


4 


901 


71 


1 


4 


901 


71 


2 


4 


901 


71 


3 


5 


399 


81 


1 


5 


399 


81 


2 


5 


399 


85 


1 


5 


399 


85 


2 


5 


399 


85 


3 


5 


901 


72 


1 


5 


901 


72 


2 


5 


901 


72 


3 


6 


399 


86 


1 


6 


399 


86 


2 


6 


399 


86 


3 


6 


399 


87 


1 


6 


901 


73 


1 


6 


901 


73 


2 


6 


901 


73 


3 


7 


901 


113 


1 


7 


901 


113 


2 


7 


901 


113 


3 



SHORT SLEP 
N LC RC SLEP ESS COHP 





H 


H 


M H 


M 


27 


17 


14 


31 1.7 


29 


94 


20 


16 


37 2.0 


33 


49 


12 


11 


23 1.8 


24 


87 


18 


14 


31 1.7 


28 


51 


21 


17 


38 2.1 


34 


50 


14 


13 


27 1.8 


26 


81 


18 


14 


32 1.7 


29 


69 


20 


18 


33 2.1 


34 


40 


14 


14 


29 1.7 


26 


74 


19 


15 


34 1.9 


31 


103 


22 


19 


41 2.9 


40 


44 


20 


17 


37 2.4 


35 


34 


22 


17 


39 2.2 


35 


56 


21 


19 


40 2.6 


38 


31 


20 


16 


36 2.3 


34 


30 


21 


17 


38 2.4 


35 


45 


22 


19 


40 2.8 


39 


11 


19 


18 


37 1.7 


32 


27 


22 


18 


40 2.4 


37 


14 


27 


22 


49 2.8 


45 


22 


27 


22 


49 3.7 


49 


48 


27 


21 


48 3.1 


46 


35 


26 


22 


47 3.1 


45 


37 


28 


23 


51 3.2 


48 


30 


25 


22 


47 3.n 


/.A 


35 


27 


22 


49 2.9 


45 


48 


29 


22 


52 3.4 


49 


43 


26 


20 


47 3.3 


46 


31 


28 


22 


50 3.1 


47 


15 


32 


27 


59 4.1 


57 


37 


31 


27 


58 3.9 
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Note. SAHP codes are 1 (Spring 1991), 2 (Fall 1991), and 3 (Spring 1992). 
* Students with all SLEP scores only. 

Number of cases read = 140 Number of cases listed = 140 
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Table C.2. Descriptive Statistics for Placement Variables, In Samples Aggregated 
By Course, Across Terms of Enrollment: By College and Subject 
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Table C.3. Correlations of Placement Variables with Course Grade, and 
Standard Deviations of Variables in Corresponding Courses, Classified 
by College and Subject (DC [263], ENG [399], ESL £400] , SPEECH [901]) 
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